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SYSTEM AND METHODS FOR CONTROLLING DEVICES BY BRAIN SIGNALS 



This invention relates generally to systems, methods and devices for 
interfacing between nerve cells (neurons) and external devices and, more 
particularly, to systems, methods and devices for extracting signals directly 
from the human brain and nervous system for use in the control of external 
devices. 

BACKGROUND OF THE INVENTION 
The human brain is an exceedingly complex processing system, which 
integrates continual streams of incoming sensory input data with stored 
memories, uses the input data and memories in complex decision processes 
at both conscious and unconscious levels, and on the basis of these 
processes generates observable behaviors by activation of its motor or 
movement control pathways and the muscles which these innervate. 

In certain cases of traumatic injury or neurological disease, however, 
the brain is partially isolated from the periphery. Input data from certain 
senses are thus lost, at least for a portion of the body, as are many voluntary 
movements. Spinal cord injury is a well known example. With spinal cord 
injury, the pathways that link higher brain regions with the spinal cord and 
that are used for control of voluntary movements may be functionally 
transected at the site of injury. As a result, the patient is paralyzed, and <s)he 
can no longer voluntarily activate muscles that are innervated by regions of 
the spinal cord below the level of the injury. Despite the injury to their long 
fibers, however, many of the cells in these higher brain regions that control 
voluntary movement will survive and can still be activated voluntarily to 
generate electric signals for controlling voluntary movement. By recording 
directly from these cells with implantable devices (e.g., electrode arrays), 
signals generated by the cells may be "exteriorized 1 ' and used for the control 
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of external prostheses, such as an assist robot or an artificial arm, or 
functional electrical stimulation paralyzed muscles. 

Another example of such loss occurs in cases of amyotrophic lateral 
sclerosis (Lou Gehrig's Disease), in which the motor neurons which control 
muscles, as well as some of the brain cells that control these motor neurons, 
degenerate. In advanced stages of this disease, the patient may have 
completely intact senses and thought processes, but is "locked in", so that 
neither movements nor behavioral expressions of any kind can be made. 
Providing these patients with some way of communicating with the external 
world would greatly enhance their quality of life. 

In sum, there is a need to develop a system for monitoring and 
processing the electrical signals from neurons within the central nervous 
system, so that the brain's electrical activity may be "exteriorized 11 and used 
for the voluntary control of external prostheses or assist devices. In this way, 
damaged pathways are circumvented and some control of the environment 
can be restored. Because the electrical fields of small groups of neurons 
drop off rapidly with distance from the cells, this system should include 
surgically implanted "tiny" electrodes or sensors, which can be placed in 
close proximity to the cells that generate command signals for voluntary 
movement. 

Earlier attempts to utilize signals recorded directly from neurons for the 
express purpose of controlling external prostheses have, however, 
encountered a number of technical difficulties. A major problem is how to 
obtain stable electrical signals of sufficient amplitude for real-time control of 
an external device. Two previous approaches have been used, but neither is 
successful in this regard. 

In the first approach, microelectrodes with small tips (< 300 em sq. 
surface area) have been used, which are positioned to within 10-100 em of a 
single neuron, thus isolating its action potentials or "spikes" from that of 
other, more distant cells. In some cases, two or three adjacent neurons are 
recorded from simultaneously. In such cases, electronic devices are used to 
discriminate between the spikes of the individual cells and to sort their "spike 
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trains" into distinctly recognizable signals. One problem with this approach, 
however, is that the effective "isolation" of the spikes of only one to a few 
neurons requires that the recording electrode be positioned in close proximity 
to the neurons. Thus, to obtain such records from a sufficient number of 
5 movement-related brain cells, scores of electrodes should be implanted in the 

hope that a few of them will be in just the right position to record signals from 
one or only a few movement controlling cells. Given the required proximity of 
the electrode and the cells, there is a high probability, however, that small 
movements of the former with respect to the latter will result in signal loss 

10 either because the electrode moves slightly away from the cells of interest, or 

closer to them, resulting in cellular injury. With blood pressure induced 
pulsations of the brain within the skull, such relative movement is not only 
possible but very likely. 

In recent years, small, multichannel, micromachined (integrated circuit) 

15 electrodes have been developed for use in neural recording. Given sufficient 

recording channel density, these electrodes promised a partial solution to the 
electrode/tissue movement problem described above. If the signal was lost 
from one channel by electrode movement, there was hope that it might be 
"picked" up by an adjacent channel, which moved closer to the active neuron 

20 at the same time that the previous one moved away. However, problems 

have been encountered with these electrodes as well. For reasons that are 
not entirely clear, neural signals are lost from these electrodes over time, due 
apparently to the formation of polarization potentials at dissimilar metal 
junctions along the recording channel, or the ensheathment and thus 

25 biological insulation of the electrode by glial cells. 

A second approach is to use electrodes with larger exposed recording 
surfaces (in the range of 0.5 to 1.5 mm sq. surface area). These low 
impedance electrodes have lower noise characteristics than those with 
smaller tips, and can reliably record the activity of hundreds to thousands of 

30 neurons at greater distances than can the latter. Indeed, low level 

electroencephalograph^ (EEG) or field potentials can even be recorded from 
the surface of the scalp. This approach thus can avoid the difficulty of 
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different signal output levels caused by small movements between the 
electrodes and the selected cells encountered in the first approach. The use 
of the signals recorded in the second approach presents, however, a major 
problem for prosthesis control. In such recordings, the desired control . 
signals may be of very low amplitude and may be "buried" within, or 
confounded by, EEG potentials from neurons that are not involved in 
voluntary motor processes. Thus, averaging must be used over many 
movement attempts to extract a usable signal. For this reason, this approach 
is less than desirable and perhaps not useful for real-time neural control of an 
external device. 

Another problem, which occurs regardless of the electrode type used, is 
that neural signals may change over time for a variety of reasons: e.g., (a) 
naturally occurring cell death, which occurs randomly throughout the brain in 
adults; or (b) learning processes, which may, over time, alter the quantitative 
relationship between a neuron's activity and the external parts of the body to 
which it contributes voluntary control. 

SUMMARY OF THE INVENTION 
The present invention overcomes some of the difficulties described 
above with improved as well as new systems, methods and devices for 
obtaining signals directly from the brain or central nervous system, and for 
processing and utilizing these signals to control external devices. The 
systems described here are adaptable to a variety of signals from the brain or 
central nervous system as diverse as a) neurally generated electrical signals, 
recorded with microelectrode technologies from within the brain or with 
surface electrodes from extracranial sites; and/or (b) measures of localized 
blood flow that are correlated with neural activity, if techniques for 
miniaturization of current devices for making such measurements, in real 
time, are developed in the future. The external devices may include any 
device that can be controlled by processed electrical signals. These devices 
include, but are not limited to, artificial or prosthetic limbs; computer 
controlled, functional electrical stimulation of muscles of paralyzed individuals 
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for the restoration of movement; robots or robotics components; computers or 
computer displays; or the teleoperation of robots and machines in hostile 
environments, 

A preferred embodiment of the invention represents a unique blend of 
technologies from the fields of neuro- or electro-physiology, biomaterials 
science, neural signal processing, functional brain imaging (to guide 
implantation of sensors), and robotics or prosthetics. Included in the 
embodiment is a unique recording arrangement with bundles of six to ten 
small (20-50 ^im in diameter), insulated, and flexible, noble metal wires that 
are arranged in a parallel or twisted array. The wire bundles are constructed 
so that each recording wire can collect multicellular signals from a small 
cluster of neurons, with tips that are incremental in length, so that many 
recording sites can be sampled along a single line of bundle insertion into the 
brain. 

According to another preferred embodiment, software routines, together 
with corresponding hardware, are used to perform specific signal correlation, 
adaptation, and distribution as part of a general recalibration procedure. A 
unique signal processing method is provided to convert recorded neural 
signals into a resultant signal that is useful for control of an external device. 
And, for the first time, the disclosed system incorporates neural net software 
routines to map actual neural signals onto desired movement functions. 

In accordance with a preferred embodiment of the invention, a robot 
arm is controlled by the neural signals recorded directly from the voluntary 
movement (motor) control areas of the cerebral cortex of a subject, such as 
an alert monkey. It can be appreciated, however, that the concepts and 
general procedures of using neural signals to control movements of a robot 
arm, as described in the various preferred embodiments of this invention, are 
valid for the control of any external device that can be manipulated directly or 
indirectly by electrical signals and are not limited to use with monkeys or 
other trained animals. The followings are the devices and stages of signal 
processing system implementation in a particular preferred embodiment of 
the invention. 

5 
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(1) Sensors (e.g., electrodes) and Sensor Arrays 

Sensors and sensor arrays used in a preferred embodiment of the 
present invention should have appropriate geometry and size for the 
configuration of the neural implantation site, and the methods of construction 
of the wire bundles used in this embodiment allow such flexibility. In the 
present invention, these sensors are comprised of miniature, multichannel 
microelectrodes, either (a) fabricated with micromachining/photolithography 
methods, or (b) from noble metal (e.g., platinum or Pt) microwires, oriented 
along a single axis in a parallel or twisted array. For example, for extraction 
of signals from the "arm" control regions of the cerebral motor cortex of a 
human for the purpose of rudimentary control of a robot arm, an estimated 
minimum of 16-24 recording channels would be needed, with an electrode 
density of about 24/cm sq. Further, the implanted electrodes and arrays 
should be tailored to the required geometry of the implantation site, so that 
the electrode recording tips contact the correct cellular regions. The precise 
locations and shapes of the correct cellular regions can be determined prior 
to implantation by magnetic resonance imaging (MRI) and other brain 
imaging procedures. Multisite recording from cells with these regions is best 
accomplished by a wire bundle electrode array, with the recording tips of the 
bundled wires located at 0.5-0.6 mm intervals along the shaft of the array. 
When inserted normal to the cortical surface and aligned with cortical lamina 
V in the anterior bank of the central sulcus, this electrode array can sample 
activities from several clusters of the cortical output neurons that would 
normally control arm/hand movements. Because the wire bundle electrodes 
are made of a single noble metal, they do not suffer the loss of signals due to 
polarization potentials at dissimilar metal interfaces, as occurs in the 
micromachined electrodes constructed to date. Moreover, because the 
exposed (non-insulated) surface area of the wire electrode recording tip (400- 
1 800 |im sq.) is larger than that typically used in an electrode designed for 
isolation of single neurons (150-300 urn sq.), signals can be recorded with 
the tips at a greater distance from the cells of interest, thus reducing the 
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probability of cellular injury by relative movements of the electrode with 
respect to brain tissue. 

(2) Electronic Microchips 

According to one aspect of the preferred embodiment, electronic 
microchips are needed for various purposes, such as amplification, filtering, 
multiplexing, and radio transmission of signals to external receivers. 
In the examples given below, preamplification of multiple channels of neural 
data, multiplexing of neural data from multiple channels into a single data 
stream, and radio transmission out of the body of a subject would be 
accomplished by chips coupled to the electrode array, that are on the order of 
2 x 2 cm sq. or less in size. 

(3) External Receivers and Demultiplexers 

According to another aspect of the preferred embodiment, external 
receivers and demultiplexers are needed for allowing the exteriorized data 
stream to be detected and reseparated into separate neural data channels, 
which can then be individually selected and mathematically transformed for 
control of specific devices (e.g. an arm prosthesis) or device components 
(e.g., separate parts of the arm prosthesis). 

(4) Signal Usage, Correlation, and Adaptation Methods 

Once the neural signals are exteriorized, the most appropriate use for 
each should be determined. For example, if signals are derived from the 
"arm" area of the motor cortex, some of the signals are related to shoulder 
movements, others to movements about the elbow, some others to 
movements about the wrist, and the rest to finger movements. (In the 
paralyzed individual, of course, the movements are "attempted" rather than 
actual voluntary movements.) If the signals are not derived from the motor 
cortex but are still under voluntary control by the subject, and are to be used 
for control of arm prosthesis movements, a determination should also be 
made as to which are the most useful signals for fine hand control, which for 
upper arm control, and so forth. 

Recorded neural signals may also change over time, due to electrode 
"drift" within the brain tissue (so that the electrode moves away from some 
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cells and closer to others), natural death of cells (50,000-100,000 cells/day in 
the adult brain), or changes in the parameters of cell discharge in relation to 
movement which result from motor learning. Consequently, a procedure 
according to the invention adapts to, or partially compensates for, changes in 
recorded neural signals and their parameters over time, so that the same 
implanted electrode array is useful for control purposes for as long as 
possible. 

A general neural prosthesis calibration procedure, including newly 
conceptualized signal correlation, adaptation, and distribution (software) 
routines, accomplishes all of these goals. In this procedure, the subject 
observes a set of simulated arm movements (or other activities) on a 
computer screen, and attempts to "track" or emulate these same movements 
with attempted movements of his/her own (paralyzed) arm. During the 
emulation process, the activity on each neural channel is cross-correlated 
with selected parameters of the simulated motion (joint rotation, position of 
the hand in space, speed of movement, etc.), to determine with which it is 
most highly correlated. The activity of this most correlated channel is then 
"routed" or "distributed" to the circuit which controls the device component or 
movement parameter with which it is most highly correlated. Since this 
calibration procedure can be performed readily by the subject or, patient on a 
periodic basis, the optimal neural channels for control of the external device 
can be reselected and redistributed each time, thus providing a continuing 
adaptation to changing neural signals, regardless of the cause of these 
changes. 

(5) Shaping of Signals for Optimal Device Control 

Once a subset of signals has been selected for control of an arm 
prosthesis (or other device), according to one aspect of the preferred 
embodiment, the signals should still be mathematically combined for optimal 
control of that device. For example, if neural signals "x and y" out of some 
larger set of signals correlate well with attempted hand movements about the 
wrist, and they are to be used for that purpose in controlling an arm 
prosthesis, they should still be mathematically processed to optimize such 
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control. By "optimal", it is meant that the processed signals should drive 
movements about the artificial wrist that match as closely as possible those 
model movements that are observed by the subject during the calibration 
procedure. To accomplish this, software networks are used for mapping 
actual neural signals onto desired movement functions; in the example 
referred to here, these would be the simulated arm/hand movements 
observed by the subject on the computer screen. 

Accordingly, it is an object of the present invention to provide an 
improved system for recording neuronal signals directly from the brain or 
central nervous system for external device control. 

It is another object of the present invention to develop an improved 
system which can be used with signals derived from small sets of selected 
single neurons, whose signals are separated from those of other nearby cells 
with electronic "spike" discrimination methods and are then recombined into a 
composite control signal. 

It is a further object of the present invention to provide an improved 
system which can also be used with signals derived from small sets of 
neurons (2-10), whose signals are simply summed and electrically processed 
to obtain a useful signal for external device control. 

It is yet another object of the present invention to provide multi- . 
electrode sensors for recording from small clusters of neurons and providing 
signals that are midway between those obtained with the single neuron and 
EEG recording approaches described above. 

It is yet a further object of the present invention to provide multichannel 
connectors and recording systems for use with the multi-electrode sensors. 

It is also an object of the present invention to provide data reduction 
methods for converting the electrical activity signals recorded from clusters of 
neurons into useful intermediate control signals. 

It is also another object of the present invention to provide an 
application of Neural Net algorithms for relating the intermediate control 
signals to movement parameters. 

9 
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It is also a further object of the present invention to provide a novel 
calibration and adaptation method for periodically rematching recorded 
signals to the desired movements of an external device, so that 
compensation occurs for fluctuating signal amplitudes and/or motor learning. 

5 

BRIEF DESCRIPTION OF THE DRAWINGS 
The accompanying drawings, which are incorporated in and form a part 
of the specification, illustrate preferred embodiments of the present invention 
and, together with description, serve to explain the principles of the invention. 
10 In the drawings: 

FIG. 1 is an exemplary diagram of a system according to the present 
invention for use by a subject with a spinal cord injury in voluntary controlling 
an external device. 

FIGS. 2 (A) to (C) are schematics of sensors (electrodes) and sensor 
15 arrays according to the present invention for recording brain activity. 

FIGS. 3 (A) and (B) are photomicrographs of recording arrays of the 
present invention used to monitor the activity of sets of cortical neurons. 

FIGS. 4 (A) to (C) are diagrams of other embodiments of electrode 
arrays and methods for bringing power to and taking neural signals from an 
20 implantable microchip. 

FIG. 5 is a diagram of an implanted device for externalizing neural 
signals for control of prosthetic devices. 

FIG. 6 is a diagram of an implanted device according to a second 
embodiment for exteriorizing neural signals. 
25 FIGS. 7 (A) and (B) are waveforms schematics of signals in the second 

stage of processing of recorded multiunit neural signals. 

FIGS. 8 is a block diagram of a second stage of processing. 
FIGS. 9 (A) to (E) are waveforms schematics of signals arising in the 
processing of multiunit recordings from the arm area of the sensorimotor 
30 cortex of a monkey. 
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FIG. 10 is a schematic of third and later stages of neural signal 
processing system, and an example of operation of the neural prosthesis for 
control of a prosthetic or a robot arm. 

FIG. 11 is a diagram of operations performed by a movement calibration 
and signal distribution system depicted schematically in FIG. 10. 

FIG. 12 is a schematic of an artificial neural net used to process neural 
input data before passing it to a peripheral device controller. 

FIGS. 13 (A) and (B) are graphs showing examples of processed neural 
discharge and the real-time prediction of wrist position from this neural 
discharge by the artificial neural net of FIG. 12. 

FIG. 14 is a flow chart showing steps used in calibrating the neural 
control system of FIG. 1, using as an example the control of an artificial limb 
or functional electrical stimulation of paralyzed muscles to produce a 
particular movement. 

DETAILED DESCRIPTION OF THE DRAWINGS 
Reference will now be made in detail to preferred embodiments of the 
invention, non-limiting examples of which are illustrated in the accompanying 
drawings. 

A. Overview 

With reference to FIG. 1, a system according to a preferred embodiment 
of the invention uses signals from cells in higher brain regions 1 can be 
activated voluntarily by a patient/subject, to in turn control external devices 
such as an artificial arm 200 and electrical stimulation of paralyzed muscles 
of an injured arm 202. FIG. 1 shows sections through various levels of the 
cerebrum 190, brain stem 192, and spinal cord 194. Also shown are 
pathways 196 that link the motor (movement control) areas of the cerebral 
cortex 198 with the spinal cord motor neurons 197 that control muscles of the 
arm 202. By recording directly from the cells in higher brain regions 1 with 
implantable devices 5 (e.g., electrode arrays), signals collected at the cells 
can, after processing by computer and electronic interfaces 9, be 
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"exteriorized" and used for the control of external prostheses, such as the 
artificial arm 200 (external pathway I) or the electrical stimulation of paralyzed 
muscles 202 (pathway II). 

Alert monkeys that are trained to make specific arm movements in order 
5 to receive small food or juice rewards are used as subjects. Under surgical 

anesthesia and sterile conditions, arrays 5 of microelectrode sensors are 
implanted into the motor (voluntary movement control) areas of the animaPs 
cerebral cortex 198, opposite the trained arm, preferably according to one of 
the methods as demonstrated in FIGS. 5-6. When the animal is fully 

10 recovered, signals from sets of neurons whose electrical activity is correlated 

in time with performance of these arm movements are identified, and the 
signals are processed mathematically for control of the robot arm as shown in 
FIGS. 7-11. This processing is such that the neural signals that are selected 
for control of a particular part of the arm (e.g., movements about the elbow) 

15 will, when applied to the controller for that part, produce robot arm . 

movements that are very similar to those made by the animal. These "off- 
line" simulations allow of adjusting neural net parameters for optimal control 
of the robot arm as shown in FIG. 12. Again using positive reward 
procedures, the animals are then trained in 1 hour long daily sessions to 

20 adapt to the movements of the nearby robot arm, which their brain signals 

now begin to move roughly in parallel with their own arm movements. 
Delivery of the food rewards is then gradually made contingent not upon the 
accuracy of the animal's own arm movements, but upon those of the robot 
arm. Eventually, the animal's own movements can be temporarily restrained 

25 (by securing the sleeve of its training jacket), and he/she can obtain food 

solely through directly controlled movements of the robot arm with the neural 
signals recorded directly from his/her brain. The systems, methods, and 
devices according to various embodiments of the current invention -can also 
be used by medical use with humans. 
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B. Sensors/Electrodes for Recording Neural Signals 

Examples of the implantable devices 5 are shown in FIGS. 2 (A) to (C). 
FIG. 2(A) shows a preferred embodiment of sensors or a sensor array 5' 
according to the present invention for recording from cortical cells of a brain 
5 for extended periods of time. Six to ten small (25 to 50 pirn diameter), 

insulated, noble metal, preferably Pt or Gold, wires 12 are bundled in a 
parallel or twisted array 10, with staggered length so that the exposed 
recording tips 14 end at different cortical depths (from the surface of the 
brain) when inserted into the brain. 

10 In one embodiment of the invention, the wires 12 are constructed of 25 

urn diameter Pt, insulated by a thin layer of Formvar, Isonel, or some other 
suitable material 15 obtained from the California Fine Wire Co. of Grover 
Beach, California. A completed wire bundle 10 made with 25 urn wire is 
approximately 75 |im in diameter. In another embodiment of the invention, 

15 the wires 12 are constructed of 50 ixn) diameter wire. A completed bundle 10 

made with 50 urn diameter wire is about 150 pirn in diameter, and is 15-35 mm 
long from electrode tip 14 to a connector assembly 50. Electrodes made with 
the 50 \irr\ diameter wire isolate the electrical activities of small clusters of 3 
or more neurons as well as those obtained with the 25 jim diameter wire. 

20 The wires 12 are held in position with respect to one another by a light- 

cured optical cement bead 16, placed 6-8 mm from a most distant wire tip 
142. The recording tips 14 of the individual wires 12 are located from 2 to 6-8 
mm distal to the bead 16. The difference in length from one tip to its neighbor 
tip is substantially equal in one embodiment of the invention, about 0.5 to 0.6 

25 mm, but these distances can be varied, depending upon the implantation site 

and its geometry. The bead 16, in one embodiment of the invention, is a 
small (0.5-0.6 mm diameter) drop of UV-curable acrylate, cured with 30 sec of 
UV exposure from a small, focused, UV gun. The cement bead 16 is also 
used for handling and manipulating the array 5* during its insertion into the 

30 brain, because of the bead's degree of rigidity. On the proximal side of the 

bead 16, the wires 18 separate to bundle loosely to the implanted connector 
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assembly 50 (see below). By inserting several staggered arrays 5* along the 
central sulcus of the cerebral cortex 198, where major motor areas exist, it is 
possible to place them all within the cortical layer that contains the 
corticospinal neurons that participate normally in the control of voluntary 
movement. MRI is used prior to surgery to locate this region precisely in 
relation to anatomical landmarks on the skull, thus guiding electrode 
implantation. Placement of eight to twenty of sensor arrays 5' in wire bundle 
form ensures recording from sufficient numbers of cells to capture a useful 
number of control signals. 

An alternative type of sensor array 5" is shown in FIGS. 2(B) and 2<C), 
where the latter is an enlarged view of recording tips 14' shown in FIG. 2(B). 
In principle, the sensor array 5" is similar to the bundled wire array 5' shown 
in FIG. 2(A), except that in this case the entire array 5" is constructed by 
micromachining or photolithographic techniques. In the embodiment shown, 
each recording channel or "wire" 12* is a very thin (4-10 urn wide) strip of 
metal, left on a silicon substrate after etching away the surrounding metal. A 
long, sharp, insulating silicone substrate 1 1 contains six to eight separate 
recording channels 146, which end in uninsulated recording pads 144 at 
various sites along a sensor shank 13 which, when covered with a bead 16' 
of epoxy or similar material, can be used for gripping and manipulation (with 
fine tools) of the sensor array 5". Again, an enlarged bead 16' or disc region 
occurs along the shank 13. On the opposite side of this bead 16", separate 
(insulated) wire channels 18' course to a 1.0 mm diameter pad 50*. Though 
more desirable because of their small size and precise geometry, long-term 
recording was less successful with the micromachined sensor array 5" than 
with the sensor array 5' in wire bundles 10 described above and shown in 
FIG. 2(A). Reasons for this include micromachined sensor's greater fragility, 
difficulty in making connections with their channels, polarization potentials 
between their dissimilar metals which block electrode conductance, the 
encasement of the electrodes by glial cells, and insulating them from neural 
signals, when they are implanted. 
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Photomicrographs of the arrays 5' and 5" are shown in "FIGS. 3 (A) and 
(B), respectively. FIG. 3(A) discloses a bundle 10 of pure platinum wires 12, 
insulated except at the tips 14. FIG. 3(B) shows a photo-lithographically 
prepared (micromachined) array 5". Thin, insulated conductors 1Z travel 
from non-insulated recording pads 14* (just visible along the shank 13 of the 
electrode 5"), up a longer cable, and to a dispersed pad where electrical 
connections can be made. 

FIGS. 4 (A) to (C) show yet other types of electrode arrays that may be 
used in the present invention. In FIG. 4(A), "n" parallel electrodes 12", each 
2.5-3.0 mm in length, protrude from a silicone wafer 20 as an insulating 
substrate, each in electrical continuity with a gold contact pad 30 on the 
opposite surface of the wafer 20. The electrodes 12" are insulated except at 
the tips 14". Each of the electrodes 12" can be made from a short length of 
90% Pt and 10% iridium wire or other noble alloy, which has sufficient 
stiffness over its short length to allow penetration of the pial covering of the 
brain and insertion into the cerebral cortex. An electrode array 25 of this type 
with each electrode 12" being a platinum plated silicon probe, but not in 
bundle form, is commercially available. As shown in FIGS. 4(B) and 4(C), an 
electrode array 25, as shown in FIG. 4(A), can be bonded to a second 
microchip 40 that contains "n" pads that mate precisely with those on the 
electrode array 25. The microchip 40 and the electrode array 25 are bonded 
in close contact and are then hermetically sealed. The microchip 40 may 
contain integrated circuits 42 that provide for amplification of the signals in 
each of the n electrodes or channels, and for multiplexing these signals into a 
serial data stream for transmission out of the brain 1 . The microchip 40 and 
electrode array 25 set upon the meninges 3 (tissue coverings of the brain 1), 
but beneath the skull (not shown), and the electrodes 12" penetrate the 
meninges 3 into the substance of the brain 1 so that neural signals can be 
directly collected therefrom. The electrodes 12" which protrude from the 
array 25 may be individual wire leads, or short bundles of the type described 
above. 
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FIGS. 5 and 6 illustrate two possible ways that the signals may be led 
from the brain 1 to external processing units. The first is a communication 
channel established by a hardware connection as shown in FIG. 5, and the 
other is a communication channel established by a wireless mechanism as 
shown in FIG. 6. 

Referring first to FIG. 5, in connection with FIG. 4<B), the implanted 
electrode array 35 has the first stage electronics circuitry 42 for signal 
processing (multichannel amplification and multiplexing) attached. Signals 
from the circuitry 42 are sent via a small, flexible cable 51 to a second stage 
device 7 implanted within the skull 2. An implant/skull junction 8 is 
hermetically sealed. This device 7 relays power from an implantable battery 
pack 56 connected to it by leads 52 beneath the skin and scalp 4. The 
implanted electronics 51 , 7 and 42 are preferably programmable and contain 
circuitry for selecting (gating on) the neural channels to be used and for 
setting other recording parameters. They also preferably contain a 
transmitter for relaying the neural signals by RF signal to a receiver, which in 
turn relays it to demultiplexing and the later stages of neural signal 
processing described in more detail in the subsequent text and figures. 

Referring now to FIG. 6, in connection with FIG. 4(C), there is shown an 
alternative implant method, according to a preferred embodiment of the 
invention, which uses a wireless mechanism, thereby allowing the cranium 
(skull) 2 to be reclosed, and thus reducing chances of infection. The 
electrode implant array 45 contains the microchip 40 which has multichannel 
amplifiers, multiplexing circuitry, and an RF transmitter. Attached to the 
microchip 40 are two coils 53 and 54 (as shown in FIG. 4(C)). One coil 53 
allows power to be transmitted to the implant array 45 via a coil in an external 
unit 47 by induction, and the second coil 54 allows transmission of the 
multiplexed, multichannel neural signal out as a serial data stream. The 
external unit 47 contains a power coil, and a chip for conversion of DC 
voltages into the AC voltages that are necessary for inductive coupling to the 
internal coil 53. All devices are implanted beneath the skin. Battery packs 56 
and the external unit 47 can be changed by simple surgical procedures if 
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necessary. With the technique shown in FIG. 6, the skull 2 is closed 
completely. While coils are used to describe the way that signals are led 
from the brain to external processing units by a nonresonant wireless 
mechanism, other kinds of devices, such as LC circuits or low pass filters, 
5 can be utilized to establish a resonant wireless mechanism to relay the 

signals as well. 

Both techniques shown in FIGS. 5 and 6 allow for complete implantation 
of the first stages of the system. In the first method (FIGS. 5 and 4(B)), a 
junction 8 is hermetically sealed between the cranial implant site 51 and the 
10 second stage device 7. In the second method (FIGS. 6 and 4(C)), the skull 2 

is resealed completely! In both cases, surgical removal and updating or 
replacement of implant components is possible if necessary. 

When these electrode arrays and devices are to be implanted in 
humans, functional MRI (fMRI) technology may first be used to identify those 
15 brain regions that the patient still has under voluntary control, thus allowing 

precise, functional placement of the implants and the recording electrodes. 

C. Second Stage Processing of Neural Signals 

The types of neural signals recorded with the electrode technologies 
described above are shown schematically in FIGS. 7(A) and (B). Typically, 
the recorded neural signals include action potentials or "spikes" (brief, 
voltage transients) which signal the discharge of small groups of cells located 
near the electrode recording tips 14. Because these cells are of different 
sizes and distances from the electrodes 12, 12\ or 12", their action potentials 
will vary in shape and amplitude, and may be separated electronically or with 
computer software on the basis of these differences. FIG. 7(A) shows, for 
example, a hypothetical voltage recorded overtime at one tip 14 of the 
multiwire bundle 10. The record shown in FIG. 9 can be seen to be 
composed of spikes of different amplitudes. At this stage, one of two major 
types of second stage processing of the recorded signals can be used. 

The first type of processing is spike discrimination method known to 
persons skilled in the art. Still referring to FIG. 7(A), an example is shown in 
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an upper trace 60 where two distinct trains 61 and 62 of action potentials, 
from two different neurons or single "units" as they are often called by 
persons skilled in the field, are interleaved in the single electrode recording. 
The single unit spike trains 61 and 62 can be separated electronically on the 
basis of differences in their spike waveforms or amplitudes, and the 
discharge frequency of each neuron (frequency = 1 /interval between spikes 
or action potentials). This separation yields two or more useful control 
signals, as shown in the lower traces 63 and 65 of FIG. 7(A), from a single 
electrode. Such spike train separation is, however, expensive in terms of 
needed hardware and software, thus raising the cost, bulk and complexity of 
any surgically implanted device. Moreover, single neurons may be damaged 
by small movements of the electrodes 12, 12\ or 12", and their signals may 
be lost. Since the brain 1 can move slightly within the skull 2, particularly in 
relation to respiratory or cardiovascular induced changes in intracranial 
pressure, such relative movements can easily occur. Thus, while the spike 
discrimination method is one in current use today by many neurophysiological 
investigators, there have been as yet no published solutions to the problems 
of mechanical recording stability. 

To circumvent the problems of mechanical recording stability, a new 
type of second-stage processing of recorded neural signals has been 
developed as demonstrated in FIG. 7(B). In this new method, the exposed 
surfaces of the recording electrode tips 14 are made slightly larger (400-1800 
^m sq.) than is optimal for the isolation of single cell discharge, so that the 
spikes of several neurons are picked up simultaneously at each electrode tip 
14. Furthermore, to reduce the possibility that the larger spikes of very 
nearby cells will dominate the record, only to be later lost through damage to 
the cells by the electrode 12, 12\ or 12", the voltage signal is converted 
logarithmically. This conversion de-emphasizes the voltage contribution to 
the multi-unit record from nearby cells, and emphasizes that from the more 
numerous, more distant cells. The signal is then electrically rectified, yielding 
a signal 64 like that shown in the top trace of panel B. This trace 64 is 
actually the log rectified representation of the trace 60 in panel A. Thus, the 
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resulting signal S(t) 64 is given by: S(t) = log|V(t)|, where |V<t)| is the absolute 
value of the recorded multiunit voltage signal 60. Note that S(t) 64 is a 
positive function of both the log amplitude and the frequency of occurrence of 
the neural spikes. It is thus a measure of the density of neural activity in the 
vicinity of the electrode tip 14, a signal that tends to be more stable over time 
than the single unit method of recording described in the first method of 
processing neural signals. 

Next, to convert this signal 64 to one that is more useful for control of 
some external device, S(t) 64 is integrated over a short time period to yield a 
signal 66 whose peak values correspond to the integral of activity over the 
preceding time interval (see FIG. 7(B)). A 33 msec integration interval is 
chosen because it is synchronized with the video recording frame duration, 
and it is found to be useful for experimental purposes. In practice, it should 
be a period of no longer than (l/2/) ( where f is the highest frequency 
component of the device movement speed that one wishes to generate. For 
human natural movement, f is on the order of 5 Hz. Thus, the integration 
period should not exceed (1/10) = 0.1 sec =100 msec. At the end of each 
integration interval (33-100 msec), the peak value 67 of the integral 66 is 
then "read" by ah electronic sample and hold (S-H) circuit, which is known to 
persons skilled in the art, the integrator is reset to zero, and another period of 
integration begins. The S-H circuit holds its value until the end of the next 
interval, and then jumps instantaneously to the new reading. Smoothing of 
the S-H output provides an analog signal 68 that is "smoother" than the 
original neural signal 60 yet still proportional to S(t) 64. Moreover, the analog 
signal 68 is much more useful for external device control because unwanted 
signals outside the frequency range of interest have been eliminated. 

An example of the second stage of signal processing is shown in FIG. 
8. During this stage of the processing, the recorded neural signals 60 having 
spikes are first converted logarithmically and rectified by the voltage 
converter 82 to yield a signal 64. Then, the signal 64 is integrated by an 
integrator 84 over a time period of no longer than (1/20 to produce an integral 
66, where the integrator 84 is connected to the voltage converter 82 as well 
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as a S-H circuit 86. The S-H circuit 86 receives the integral 66 t reads the 
peak value of the integral 66, and provides an analog signal 68 after 
smoothing. 

As an illustration, an example of these operations performed upon 
actual neural recordings from the brain of an alert monkey is shown in FIGS. 
9(A) to (E). The trace in FIG. 9(A) shows the log amplitude of the multi-unit 
activity 160 recorded from a cluster of neurons in the sensorimotor cortex of 
the monkey, during flexion (161-163 ) of his elbow as shown in FIG. 9(E); 
note the increased activity during elbow flexion 162. The trace 164 in FIG. 
9(B) shows the absolute or rectified value of the signal 160. The trace 166 as 
shown in FIG. 9(C) shows the output of the short interval, resettable 
integrator, and the trace 168 as shown in FIG. 9(D) shows the output of the 
S-H circuit that "reads" the value of the integrator at the end of each short 
integration period. The recordings are from experiments with a trained 
monkey, carried out during development of the technologies disclosed here. 
The time/voltage calibrations are 0.1 sec and 50 microvolts. 

D. Third and Later Stages of Signal Processing 

A summary of the third and subsequent stages of signal processing is 
shown in FIG. 10 according to a preferred embodiment of the present 
invention. 

The processed signals 68 from the S-H circuit 86 are digitized by an 
analog/digital (A/D) signal converter 100 and then fed in parallel to (a) a 
multi-channel display system 102, and (2) a signal /movement cross 
correlation and distribution system 101. The display system 102 is an n-cell 
matrix, where "n" is an integer and each cell 104 in the matrix 102 represents 
a separate neural data channel. The system 102 provides a quick overview 
of the pattern of activity across the "n" selected neural recording channels for 
each of the integration periods described above, and is extremely useful for 
initially 'focusing in" on those channels that are most active during particular 
actual or attempted movement. The display system 102 includes a LED 
display, where the intensity of the LED lighting is a positive function of the 
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level of neural activity in that channel, or a small computer screen which 
displays the ongoing level of activity in each channel in the form of color- 
coded values. Typically, the range of neural activity is divided into eight 
levels for monitoring purposes, though it is actually digitized for subsequent 
processing with 12-bit accuracy. 

The signal/movement cross correlation and distribution system 101 is 
preferably a X" language software routine which performs the following 
operations as illustrated in FIG. 1 1 according to a preferred embodiment of 
the present invention: 

(1) Determination of mean signal levels 1 12 on each channel when the 
subject is at "rest"; 

(2) Subtraction of the mean resting signal level 112 for each channel 
from the ongoing signal 1 10 of that channel, thus yielding a time-dependent 
signal D(t) 114 that departs from zero only when a movement is attempted; 

(3) Computation of the cross correlations between D(t) 114 and the 
computerized display 116 of model arm (or joint) position, velocity, or 
acceleration (see the correlation table in step 2, FIG. 1 1 ) as observed by the 
subject during the calibration procedure described in detail below; 

(4) On the basis of these correlations, selection of the channels 118 
that will be most useful for control of movements or movement-related 
parameters of the external device; 

(5) Routing/distribution of the selected channels 1 18 to the artificial 
neural nets (shown in FIG. 12) that will "shape" that signal for control of 
particular movements or of any other external device parameter. 

Refer now back to FIG. 10. After selection of the subset 1 18 of 
channels that will be used to control a particular part of the external device, in 
this example movements about one of the "joints" in a robot arm, the signals 
on the channels 118 are fed to a three-layer, software or Artificial Neural Net 
(ANN) 103 for further processing, before passing to a peripheral device 
controller 107. After a succession of attempted or actual movements 105, in 
which each ANN 103 learns to "map" selected neural data channels 118 onto 
some measured output parameter, the ANNs 103 will thereafter shape the 
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input signals appropriately for device control. In the case of the paralyzed 
individual, the output parameter would be a desired or "calibrated 1 ' movement 
of the arm, or movement about a major joint in the arm, that (s)he observes 
on a computer display and "tries" to emulate with his/her own paralyzed arm. 
This "calibration" procedure is described in more detail in the next section. 

The properties of a current ANN 103 used for this purpose are depicted 
in FIG. 12. The preprocessed neural signals 120, with resting back-ground 
levels removed, are fed from the particular neural channels 118 selected for 
control of wrist movements of a robot arm to each input node of a software 
ANN 103. The ANN 103 has a standard three layer, feedforward network that 
is trained using back propagation with momentum. The first layer contains 
three input nodes 122, the second, hidden layer contains five nodes 124, and 
the output layer has a single node 126 (scalar output). All of the nodes are 
fully interconnected in a strictly feedforward manner; i.e., each node in a layer 
is fully interconnected in a feedforward manner to every node in the next 
layer, and only to those in the next layer. For instance, each input node 122 
receives input data 120 from all of the selected channels 118, it then sends 
an output to each node 124 in the middle or hidden layer. Each node 124 in 
the hidden layer receives inputs from each of the nodes 122 in the input 
layer. In turn, all nodes 124 in the hidden layer send outputs to the single 
node 126 in the final or ANN output layer. To further smooth out data 
transients that are outside of the control frequencies of interest (0-5 Hz), each 
data point fed to the ANN 103 comprises a running average of the current 
value of the neural signal in channel "m" (at time T) plus the previous nine 
values of that signal (from time t-1 to t-9). This averaging period was 
determined by trial and error and may be changed to other values. 

An example of the success of this three-layer net in mapping two 
channels of neural data onto a simultaneously obtained measure of an alert 
monkey's wrist position, according to a preferred embodiment of the present 
invention, is shown in FIGS. 13(A) and 13(B). Referring first to FIG. 13(A), 
top two traces 130 and 132 show processed multiunit activity (outputs of S-H 
circuits) from two channels, identified as channel 3 and channel 14 
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respectively, of a 36-electrode array implanted in the left motor cortex of a 
trained monkey, recorded as the animal flexed and extended his right wrist. 
Actual wrist position is shown by a third trace 134. Channel 3 shows a peak 
change in activity leading wrist extension, with a smaller elevation during 
wrist flexion. Channel 14 shows a large increase in activity leading and 
during wrist flexion, and a moderate increase during extension. Referring 
now to FIG. 1 3(B), output 1 38 of the ANN 1 03, which in this case is the 
position of the animal's wrist predicted from activity in neural channels 3 and 
14, is shown by the heavy trace adjacent to an offset lighter movement trace 
136 which shows his actual wrist position. Note the good correspondence, 
even though only two neural channels were used to compute predicted wrist 
position 138. The movement trace 136 is not the same as that shown in the 
previous trace 134, but was computed and recorded later, during training of 
the ANN 103. After the ANN 103 was trained over 10 movement trials, the 
match between actual and predicted wrist position attained a correlation of r « 
0.96. Use of the ANN 103 output 138 to drive a robot ,t wrist" produced a 
robot wrist movement trace indistinguishable from the ANN 103 output 138. 

Finally, referring back to FIG. 10, the output 138 of each neural net 103 
is fed to an interface controller 107. An interface controller is a device which 
converts the output signals from one system into an appropriate set of signals 
for controlling some other system or device and known to persons skilled in 
the art. Interface controller 107 comprises a microprocessor which is 
programmed to convert the output of the neural net 103 into the voltages and 
currents that is necessary for actuating, for example, the robot controller, and 
thus moving the robot's wrist. 

E. Calibration and Use of the System 

FIG. 14 shows steps that are followed by a paralyzed individual in 
setting up, calibrating, and using a neurally control external device according 
to a preferred embodiment of the present invention. Here the control of an 
artificial limb, or of stimulation of paralyzed muscles of the limb, to produce a 
particular motion is used as a particular example. It is assumed in this 
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description that two prior conditions have been met. First, the electrodes 
have been implanted in brain regions that have been shown with functional 
imaging procedures to be under the subject's voluntary control; i.e., which are 
"activatable" by the subject when desired. If possible, they should also be 
implanted in a brain region that is normally activated when the subject 
attempts one or more of the particular movements that the prosthetic device 
will be asked to generate, though this is not an absolutely necessary 
requirement. Second, a prosthetic specialist will have already set channel 
gains and other parameters after device implantation. Each day or at 
selected intervals thereafter, the patient/subject would perform the following 
calibration routine. 

First, (s)he learns to use a particular, monitored biological signal to turn 
on the system calibration computer when desired. If eye movements are 
intact, this signal could be a patterned sequence of eye blinks, sensed by a 
small monitoring device, that would not occur normally. If control of certain 
neck or facial muscles remains, their electromyographic activity could be 
used in the same coded way. In a totally paralyzed patient, the signal could 
be a particular time code of brain activity, generated voluntarily by the patient 
but unlike that which would occur during normal operation of the external 
device. A similar set of signals can be used for turning the calibration system 
off, or later, for turning the device to be controlled on or off when the neural 
control system has been calibrated. 

Second, with the assistance of another person or again using a coded 
sequence of biological signals, the subject selects a particular movement for 
calibration from a predetermined list. The list may have the following options: 
alternate flexion and extension of the elbow; flexion-extension of the wrist; 
movements about the shoulder; grasping and releasing of an object; reaching 
to different points in space; or some combination of these. 

When the subject gives a "ready" signal, the selected movement is 
displayed on a video monitor, simulated by animation or a video record of a 
model performing an actual movement. The model movement is performed at 
a slow to moderate speed and, during its performance, the subject "tracks" 
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the observed movement by attempting to move his/her own paralyzed limb in 
exactly the same manner and at the same speed. "N/2" repetitions would be 
performed, where N is a non-zero even integer. During these repetitions, the 
system (a) cross correlates the neural signals with the model movement(s) or 
5 some selected parameter (e.g., position, velocity, acceleration) of that 

movement; and (b) on the basis of the average of these correlations over the 
N/2 repetitions, determines which subset of neural channels was most highly 
correlated with the model movement and its parameters. The activity on the 
selected subset channels would then (c) be routed to the ANNs 103 which 

10 control the components of the external device that will produce a movement 

like the model movement. Another N/2 repetitions are then performed, and 
the appropriate ANNs 103 (d) "map" these selected neural inputs onto a 
stored record of parameter values for the model movement. 

The subject now performs this same procedure for the next movement 

15 in the set, and the four steps (a) through (d) are repeated for that movement. 

And so on, until the entire calibration procedure has been performed. This 
procedure is performed daily or only at needed intervals (when the subject 
notices a diminution in control accuracy), with the optimal neural channels 
being re-selected at each calibration procedure and mapped again onto the 

20 desired movement functions. Thus, this periodic calibration procedure 

adapts to or allows compensation for changing neural signal parameters, and 
it ensures the optimal selection of those channels that are still useful at any 
time for device control. 

Please note also that the subject's own brain can also adapt to 

25 changing signal properties and the challenges that these changes impose on 

device control. That is, if the subject can voluntarily activate the brain region 
from which signals are monitored and can vary these signal levels, then (s)he 
can learn to modulate these activation levels so that the external device can 
still be manipulated, even if there is drift or other unknown changes in the 

30 activity of the recorded neural channels. 

Obviously, many modifications and variations of the present invention 
are possible to persons skilled in the art, without departing from the spirit and 

25 



NSOOCID: <WO 0009008A1J_> 



WO 00/09008 PCT/US99/18172 

the scope of the invention disclosed herein. It is to be understood, therefore, 
that the invention can be practiced otherwise than as specifically described. 



<WO 0009008A1J_> 



26 



WO 00/09008 



PCT/US99/18172 



CLAIMS 

What is claimed is: 

1 . A device for collecting multicellular signals directly from a nervous 
system and transmitting the signals to an external receiver, comprising: 

A) a plurality of electrodes formed in bundles of flexible wires with 
tips at staggered length aligned to be implanted in the nervous system for 
collecting multicellular signals; and 

B) a signal processing mechanism connected to the electrodes for 
multiplexing and transmitting the signals from the electrodes to the external 
receiver. 

2. The device according to claim 1 , wherein the number of the flexible 
wires in a bundle ranges from 3 to 10. 

3. The device according to claim 1 , wherein the diameter of the flexible 
wires is smaller than 50 em. 

4. The device according to claim 1 , wherein the difference in length from 
one tip to its neighbor tip of the wires is variable. 

5. The device according to claim 4, wherein the difference in length from 
one tip to its neighbor tip of the wires is about 0.3 to 0.6 mm. 

6. The device according to claim 1 , wherein the flexible wires in a bundle 
are made from noble metal. 

7. The device according to claim 1, wherein the signal processing 
mechanism comprises: 

A) an insulating substrate; 

B) an array of electrode contact pads mounted on the substrate for 
receiving the electrodes, each electrode received individually by one of the 
electrode contact pads, and with the electrode contact pads being electrically 
isolated from each other at the substrate; 

C) a microchip bonded in close contact with the substrate for 
receiving and multiplexing the signals from the electrodes; and 

D) an electric connection to the microchip for relaying power to the 
microchip and transmitting the signals received at the microchip to the 
external receiver. 
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8. The device according to claim 7, wherein the electric connection is a 
hardware device. 

9. The device according to claim 8, wherein the hardware device is a 
cable. 

10. The device according to claim 7, wherein the electric connection is a 
wireless device. 

1 1 . The device according to claim 10, wherein the wireless device is a 
nonresonant wireless setup. 

12. The device according to claim 1 1 , wherein the nonresonant wireless 
setup comprises: 

A) a first coil in connection with the microchip for relaying power to 
the microchip; and 

B) a second coil in connection with the microchip for receiving and 
transmitting the multiplexed signals to the external receiver. 

13. The device according to claim 10, wherein the wireless device is a 
resonant wireless setup. 

14. The device according to claim 13, wherein the resonant wireless setup 
comprises: 

A) a coil in connection with the microchip for relaying power to the 
microchip; and 

B) a low pass filter in connection with the microchip for receiving 
and transmitting the multiplexed signals to the external receiver. 

15. The device according to claim 7, wherein the microchip comprises 
signal processing integrated circuits. 

16. An apparatus for recording multi-neuron signals directly from a nervous 
system comprising a plurality of sensors, wherein each sensor further 
comprises a bundle of noble metal wires with tips at staggered length. 

17. An apparatus for recording multicellular signals directly from small 
clusters of neurons and broadcasting the signals to an external receiver, 
comprising: 

A) an array of electrodes for receiving the signals; 
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B) a signal processor for multiplexing and transmitting the signals 
to the external receiver, wherein the signal processor comprises: 

i) an insulating substrate; 

ii) an array of electrode contact pads mounted on the 
substrate for receiving the electrodes, each electrode received individually by 
one of the electrode contact pads with the electrode contact pads being 
electrically isolated from each other at the substrate; 

iii) a microchip bonded in close contact with the substrate 
for receiving and multiplexing the signals from the electrodes, wherein the 
microchip comprises signal processing integrated circuits; and 

iv) a electric device connected to the microchip for relaying 
power to the microchip and carrying the signals from the microchip to the 
external receiver. 

18. The apparatus according to claim 17, wherein the electrodes are formed 
in a bundle of noble metal wires with tips at staggered length. 

19. The apparatus according to claim 17, wherein the electric device is a 
hardware connection. 

20. The apparatus according to claim 19, wherein the hardware connection 
is a cable. 

21 . The apparatus according to claim 17, wherein the electric device is a 
wireless connection. 

22. The apparatus according to claim 21 , wherein the wireless connection 
is a nonresonant wireless setup. 

23. The apparatus according to claim 22, wherein the nonresonant wireless 
setup comprises: 

A) a first coil in connection with the microchip for relaying power to 
the microchip; and 

B) a second coil in connection with the microchip for receiving and 
transmitting the multiplexed signals from the microchip to the external 
receiver. 

24. The apparatus according to claim 21, wherein the wireless connection 
is a resonant wireless setup. 
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25. The apparatus according to claim 24, wherein the resonant wireless 
setup comprises: 

A) a coil in connection with the microchip for relaying power to the 
microchip; and 

B) a low pass filter in connection with the microchip for receiving 
and transmitting the multiplexed signals from the microchip to the external 
receiver. 

26. An instrument for collecting multicellular signals directly from a nervous 
system and transmitting the signals to an external receiver, comprising: 

A) at least one sensor for receiving the signals, wherein the sensor 
comprising: 

i) a bundle of wires with staggered length in an array, 
wherein each wire has a tip for receiving the signals from the nervous system; 
and 

ii) an optical cement bead through which the wires may 
pass for holding the wires in position in a selected distance from the most 
distant wire tip; and 

B) a signal processor connected to the sensor for multiplexing and 
transmitting the signals from the sensor to the external receiver. 

27. The instrument according to claim 26, wherein the selected distance is 
about 6-8 mm. 

28. The instrument according to claim 26, wherein the wires are made from 
conductive materials. 

29. The instrument according to claim 28, wherein the conductive materials 
are noble metal. 

30. A method for using multicellular signals collected directly from a 
nervous system to control an external device comprising the steps of: 

A) collecting multicellular signals directly from a nervous system; 

B) multiplexing and broadcasting the signals to an external 
receiver; 

C) receiving the signals for demultiplexing and separating the 
signals; 
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D) rectifying the signals; and 

E) converting the signals logarithmically. 

31 . The method according to claim 30, further comprising the steps of: 

F) integrating the log rectified signals over a short time period; 

G) sample-hold reading of the peak value of the integral for 
generating a S-H output; and 

H) using the S-H output for controlling the external device. 

32. A channel display system for monitoring recorded multicellular signals, 
comprising: 

A) a device for recording multichannel neural signals directly from 
a nervous system and broadcasting the signals; 

B) an external receiver for receiving the signals; 

C) electric connection means for connecting to an external receiver 
to relay the signals; and 

D) a visual display for displaying the signals in a matrix, wherein 
each cell of the matrix represents a separate neural data channel. 

33. The channel display system according to claim 32, wherein the visual 
display comprises an LED, wherein the intensity of illumination of the LED is 
a positive function of the level of neural signals fed from the related neural 
data channel. 

34. The channel display system according to claim 32, wherein the visual 
display comprises a computer screen, wherein the color of each cell of the 
display matrix on the screen is coded according to the level of neural signals 
fed from the related neural data channel. 

35. A method for using multicellular signals collected directly from a 
nervous system to control a prosthetic device comprising the steps of: 

A) using a detective device with a plurality of electrodes to record 
multicellular signals directly from a nervous system; 

B) multiplexing and broadcasting the signals to an external 
receiver; 

C) processing the signals; 

D) converting the signals in digital form; and 
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E) displaying the signals in a multi-channel visual display where 
the display in one channel correlates to signals collected from one electrode. 

36. A method for using multicellular signals collected directly from a 
nervous system to control a device comprising the steps of: 

5 A) setting up a calibration routine for a subject's learning of a 

movement; 

B) developing a model for that movement; 

C) using a detective device with a number of electrodes to record 
multicellular signals directly from the subject's nervous system, wherein one 

10 electrode represents one signal channel; 

D) multiplexing and broadcasting the signals to an external 
receiver; 

E) processing the signals; 

F) converting the signals in digital form; 

15 G) determining of mean signal levels over a period of 10 to 20 

seconds for each channel where signals in one channel correlates to signals 
collected from one electrode when the subject is at rest; 

H) subtracting the mean resting signal level for each channel from 
the signals recorded at different time to yield a signal as a function of time, 

20 D(t), that departs from zero only when a movement is attempted; 

I) computing the cross correlation between D(t) and the 
computerized model of the attempted movement in terms of at least one 
motion parameter; and 

J) selecting the neural channels which are useful for controlling 
25 the device or a component of the device based on the cross correlation 

calculation. 

37. The method according to claim 36, further comprising the step of: 

K) distributing the selected channels to neural nets for shaping the 
signals for control of the movement of the device. 
30 38. The method according to claim 36, wherein the motion parameters are 

position, velocity, and acceleration. 
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39. An apparatus for collecting multicellular signals directly from a nervous 
system to control an external device, comprising: 

A) sensors in an array for recording the signals; 

B) a transmitter for broadcasting the signals; 

C) a processor for receiving the signals, converting the signals 
logarithmically, and integrating the signals to generate a S-H output in analog 
form; 

D) a converter for converting the signals in analog form into 
digitized signals; 

E) a computing interface for performing neural signal/model 
movement cross correlation to select the signals that are most useful for 
control of a particular movement of the device and distributing the selected 
signals; 

F) a microcomputer for supporting a feedforward neural network 
for receiving the selected signals and mapping the signals onto a model 
movement to generate control parameters; and 

G) an interface controller for controlling the device to make 
movement same or similar to the model movement according to the control 
parameter. 

40. A method for using a neural net to map neural signals onto desired 
movement or control parameters to control a device comprising the steps of: 

A) recording neural signals in multichannels directly from a 
subject's nervous system; 

B) removing resting background levels from the signals; 

C) calculating a running average for each data point of the signals 
at time T in channel "m" by combining the previous several values of that 
signal; 

D) feeding the signals from the particular channels selected 
according to the desired movement or control parameters in a feedforward 
direction to each input node of the neural net, wherein the neural net has at 
least three input nodes, and all the nodes are fully interconnected; and 
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F) outputting the signals from the neural net to a peripheral device 
controller. 

41 . A method for calibrating a neutral control system comprising the steps 
of: 

5 A) turning on the control system; 

B) selecting a model movement for calibration; 

C) giving ready signal to the system; 

D) displaying the selected model movement on a visual display; 

E) allowing a subject to attempt to track or emulate the selected 
10 model movement; 

F) recording neural signals from a number of neural channels of 
the subject during the attempted movement; 

G) comparing the recorded neural signals with the selected model 
movement by correlation methods, so that the neural channels whose activity 

15 is most highly correlated with the attempted movement may be selected by 

the system; 

H) repeating steps E) to G) for N/2 times, wherein N is a number 
equal to or greater than one; and 

I) performing another N/2 repetitions of E) to G) to allow a neural 
20 net to map the recorded neural signals from the selected channels onto a set 

of parameters for the selected model movement. 

42. A movement calibration system comprising: 

A) a biological signal monitor for turning on or off the system; 

B) a database for storing optimal parameters related to a family of 
25 model movements a subject attempts to make; 

C) a visual display for displaying a movement selected from the 
database; and 

D) a computer for matching the selected movement and the 
subject's attempted movements by calculating their correlation. 

30 43. A method for using multicellular signals collected directly from a 

nervous system to control a prosthetic device comprising the steps of: 
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A) setting up a calibration routine for a subject's learning of a 
desired movement; 

B) computerizing a "model" for that desired movement; 

C) using a detective device with a number (n) of electrodes to 
record multicellular signals directly from the subject's nervous system when 
the subject attempts the desired movement, wherein each of the n electrodes 
represents one signal channel; 

D) multiplexing and broadcasting the signals to an external 
receiver; 

E) processing the signals for generating signals in analog form; 

F) converting the signals into digital form; 

G) determining of mean signal levels for each channel where 
signals in one channel correlates to signals collected from one electrode 
when the subject is at "rest"; 

H) subtracting the mean resting signal level for each channel as 
computed in step (G) from the collected ongoing signal to yield a signal as a 
function of time, D(t), that departs from zero only when a movement is 
attempted by the subject; 

I) computing the cross correlation between D(t) and the 
computerized model of the desired movement in terms of kinetic parameters 
such as position, velocity, and/or acceleration; 

J) selecting the neural channels which are most useful for 
controlling the device or a component of the device based on the cross 
correlation calculation; and 

K) distributing of the selected channels to neural nets to shape the 
signals for controlling the device to produce a movement as same or similar 
to the desired movement. 



35 



_0009008A1J_> 



WO 00/09008 



PCTAJS99/18172 



1/14 




NSOOCW> <WO 0009008A1_L> 



WO 00/09008 



PCT/US99/18172 




NSOOCID: <WO 0009006A1_I_> 



WO 00/09008 



PCTAJS99/18172 



3/14 




Ff5 



^SDOCIO: <WO 0009006A1_I_> 



WO 00/09008 PCT/US99/18172 



4/14 





T^4 



WSOOCID: <WO 0009008A1_U> 



WO 00/09008 



PCT/US99/18172 




WO 00/09008 



PCT/US99/18172 



6/14 




WO 00/09008 



PCT/US99/18172 



7/14 



t 



Voltage 



t 



Discharge 
Frequency 



Time 



B 




Multiunit Recording 




J 



Discharge Frequency of Unit 1 



Discharge Frequency of Unit 2 ^(j- 





Logarithm of rectified multiunit signal 

64 



iMiiiliU i ikiii i\i i It k It l 



/I 



Integral over short time span of 
log rectified signal 



Voltage 



Time 



Reading of peak integrals by 
sample and hold Circuit 

U 



^•7 



JNSOOC1D: <WO 0009008A1J_> 



WO 00/09008 



PCT/US99/18172 





5" 



CM 
00 




3NSDOCID: <WO O0O900BA1 J_> 



WO 00/09008 



PCT/US99/18172 



A 9/14 



Log multiunit recording 1^ 




WO 00/09008 



PCT/US99/18172 



10/14 



) 









A/D 
CONVERSION 


► 


□□CO 


► 

► 


□□□□ 


• 
• 

► 


□□□□ 


□□□□ 



SIGNAL/COMPONENT CROSS CORRELATION 
& 

DISTRIBUTION SYSTEM 



i 0 > NEURAL 
1 NETS 



loir 



i i i 



i 
i 

T 



l°7 



INTERFACE/ 



SHOULDER 




ELBOW 




WRIST | 




r 






j SHOULDER | 




ELBOW j 




WRIST j 



■ HAND POSITION I 
IN SPACE | 



t — r — r- J 
L [ i 



i 



HAND 
GRASP 



HAND 
GRASP 




WSDOCID: <WO 0009008A1J_> 



WO 00/09008 



PCT/US99/18172 



11/14 



STEP 1 



Neural Signal 
at Rest 



Signal 
Level 



0 



Time 



Compute and Subtract 
Mean Signal at Rest 



Signal Out 
to Next Stage 



11-4- 



STEP 2 



N 
E 
U 
R 
A 
L 

C 
H 



ATTEMPTED MOVEMENT life 
12 3 . . 



1 

2 
3 

4 

5 
6 



0.8 


0.2 


-0.7 




-0.2 


0.8 


0.0 




0.7 


0.4 


0.5 




0.6 


-0.3 


0.9 




0.0 


0.7 


-0.1 




0.1 


0.6 


0.5 





CHANNELS , . n 
MOVEMENT SELECTED ' 'O 



1,3,4 
2,S,6 



1,4 



VSOOCIO: <WO 0009008A1J_> 



WO 00/09008 



PCT/US99/18172 



12/14 




NSOOCID: <WO 0009008A1J_> 



WO 00/09008 



PCT/US99/18172 



1 3/14 



14 



Neural 
Channel 




Extended 



t 



Wrist 
Position 



Flexed 



Wrist r\ 
Position 




10 sec 



ANN 
Output 




Ftf. 13 



MSOOCID: < WO 0009008A 1 _L> 



WO 00/09008 



PCT/US99/18172 



14/14 



SYSTEM CALIBRATION METHOD 




TURN ON CALIBRATION SYSTEM 
SELECT MOVEMENT FOR CALIBRATION (a,b, . . ) 

+ 

GIVE READY SIGNAL ~* 

ATTEMPT TO "TRACK" MOVEMENT 
•N' MOVEMENTS ? 

/\ 

YES NO 

♦ 

LAST MOVEMENT TO BE CALIBRATED? 

/\ 

YES NO : 

END CALIBRATION 



NEXT 
MOVEMENT 



JNSDOCIO: <WO 0O09O08A1_L> 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US99/I8I72 



A. CLASSIFICATION OF SUBJECT MATTER 
1PC(6) :A61B 5/04 
US CL :600/372. 378 

According to International Patent Classification (IPC) or to both national classification and IPC 



FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 
U.S. : 600/372, 373. 377, 378, 383, 544, 546; 607/48, 49, 62. 1 16,-1 18; 606/130 



Documentation searched other than minimum documentation to the extent that such documents arc included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



x 

Y 

X 
Y 
X 



US 5,178,161 A (KOVACS) 12 January 1993, entire document. 



US 5,524,338 A (MARTYNIUK et al.) 11 June 1996, entire 
document. 



US 5,413,103 A (ECKHORN) 09 May 1995, entire document. 



7-9, 15, 17, 19, 
20, 35 

10-14, 16, 18, 21- 
25, 36, 37, 42 

1-6, 26, 28, 29 

16, 18, 27 

1-6 



| x| Further documents are listed in the continuation of Box C. | j See patent family annex. 



•E' 

'1/ 



Special categoric* of cited documents: 

document defining the general ttate of the ait which is not considered 
to be of particular relevance 

earlier document published on or after the international Tiling date 

document which may throw doubts on priority claim (s) or which is 
cited to establish the publication date of another citation or other 
special reason (as specified) 

document referring to an oral disclosure, use, exhibition or other 
means 

document published prior to the international filing date but later than 
the priority date claimed 



"T* later document published after the international filing date or priority 

date and not in conflict with the application but cited to understand 
the principle or theory underlying the invention 

"X* document of particular relevance; the claimed invention cannot be 

considered novel or cannot be considered to involve an inventive step 
when the document is taken alone 

*Y" document of particular relevance; the claimed invention cannot be 

considered to involve an inventive step when the document is 
combined with one or more other such documents, such combination 
being obvious to a person skilled in the art 

"A" document member of the same patent family 



Date of the actual completion of the international search 
05 NOVEMBER 1999 



Date of mailing of the international search report 

2 3 DEC 1999 



Author 



Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box PCT 

Washington, D C. 20231 
Facsimile No. (703) 305-3230 



AuiMorize'd ofltcej_,. 



DAVID RUDlAY 
Telephone No. (703) 308-3595 



Form PCT/iSA/210 (second sheetXJuly 1992)* 



>*SDCCID: <WO_ 



INTERNATIONAL SEARCH REPORT 



Intern, jnal application No. 
PCTAJS99/I8172 



C (Continuation). DOCUMENTS CONSIDERED TO BE RELEVANT 



Category" 



A 
A 



US 5,279,305 A (ZIMMERMAN et al.) 18 January 1994, entire 
document. 

US 5,638,826 A (WOLPAW et al.) 17 June 1997, entire document. 

US 5,215,088 A (NORMANN et al.) 01 June 1993, entire 
document. 

US 5,692,517 A (JUNKER) 02 December 1997, entire document. 



Citation of document, with indication, where appropriate, of (he relevant passages 



Relevant to claim No. 



10-14, 21-25 

1-43 
1-43 

1-43 



Form PCT/ISA/210 (continuation or second sheet )(July 1992)* 

MSOOCIO: <WO C009008A1 J_> 



CORRECTED 
VERSION* 



WORLD INTELLECTUAL PROPERTY ORGANIZATION 
Internationa) Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY <PCT) 



(51) International Patent Classification 6 : 
A61B 5/04 



Al 



(11) International Publication Number: WO 00/09008 

(43) International Publication Date: 24 February 2000 (24.02.00) 



(21) International Application Number: PCT/US99/ 1 8 1 72 

(22) International Filing Date: 1 1 August 1999 (11 .08.99) 



(30) Priority Data: 

09/135,249 



17 August 1998 (17.08.98) 



US 



(71) Applicant: EMORY UNIVERSITY [US/US]; 1380 South 
Oxford Road. Atlanta, GA 30322 (US). 

(72) Inventor: HUMPHREY, Donald. R.; 2617 Woodwardia Road, 
N.E., Atlanta, GA 30345 (US). 

(74) Agents: PRATT, John, S. et al.; Kilpatrick Stockton LLP, Suite 
2800. 1 100 Peachtree Street, Atlanta. GA 30309-4530 (US). 



(81) Designated States: AU, CA, JP, European patent (AT, BE, 
CH. CY, DE, DK, ES, FI, FR, GB. GR, IE, IT, LU, MC. 
NL, PT, SE). 



Published 

With international search report. 

Before the expiration of the time limit for amending the 
claims and to be republished in the event of the receipt of 
amendments. 



(54) Title: SYSTEM AND METHODS FOR CONTROLLING DEVICES BY BRAIN SIGNALS 




(57) Abstract 



A system and method control prostheses (200), other devices with signals received by sensors, implanted directly in the brain or 
other parts of the nervous system of a subject, and transmitted to an external receiver. Included in the system are sensors T5) in the form 
of bundles of small, insulated, flexible wires (10), configured in a parallel or twisted array, which are used to receive multicellular signals 
from small clusters of neurons. A new "calibration/adaptation" system is developed, in which the neural signals are cross-correlated with 
the parameters of a set of standardized or model movements as the subject/patient attempts to emulate the model movements, and on the 
basis of the correlations the neural signals that are best suited for control of the corresponding movement or movement parameter of the 
external devices are selected. 



•(Referred to in PCT Gazette No. 46/2000, Section II) 

NSDCCID: <WO 00O90OBA1 JA> 



FOR THE PURPOSES OF INFORMATION ONLY 
Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



AL 


Albania 


ES 


Spain 


LS 


Lesotho 


SI 


Slovenia 


AM 


Armenia 


Fl 


Finland 


LT 


Lithuania 


SK 


Slovakia 


AT 


Austria 


FR 


France 


LU 


Luxembourg 


SN 


Senegal 


AU 


Australia 


GA 


Gabon 


LV 


Latvia 


sz 


Swaziland 


AZ 


Azerbaijan 


GB 


United Kingdom 


MC 


Monaco 


TD 


Chad 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


MD 


Republic of Moldova 


TG 


Togo 


BB 


Barbados 


GH 


Ghana 


MG 


Madagascar 


TJ 


Tajikistan 


BE 


Belgium 


GN 


Guinea 


MK 


The former Yugoslav 


TM 


Turkmenistan 


BF 


Burkina Faso 


GR 


Greece 




Republic of Macedonia 


TR 


Turkey 


BG 


Bulgaria 


HU 


Hungary 


ML 


Mali 


TT 


Trinidad and Tobago 


BJ 


Benin 


IE 


Ireland 


MN 


Mongolia 


UA 


Ukraine 


BR 


Brazil 


1L 


Israel 


MR 


Mauritania 


UG 


Uganda 


BY 


Belarus 


IS 


Iceland 


MW 


Malawi 


US 


United States of America 


CA 


Canada 


IT 


Italy 


MX 


Mexico 


uz 


Uzbekistan 


CF 


Central African Republic 


JP 


Japan 


NE 


Niger 


VN 


Viet Nam 


CG 


Congo 


KE 


Kenya 


NL 


Netherlands 


YU 


Yugoslavia 


CH 


Switzerland 


KG 


Kyrgyzstan 


NO 


Norway 


ZW 


Zimbabwe 


CI 


Cdte dMvoire 


KP 


Democratic People's 


NZ 


New Zealand 






CM 


Cameroon 




Republic of Korea 


PL 


Poland 






CN 


China 


KR 


Republic of Korea 


PT 


Portugal 






cu 


Cuba 


KZ 


Kazakstan 


RO 


Romania 






cz 


Czech Republic 


LC 


Saint Lucia 


RU 


Russian Federation 






DE 


Germany 


U 


Liechtenstein 


SD 


Sudan 






DK 


Denmark 


LK 


Sri Lanka 


SE 


Sweden 






EE 


Estonia 


LR 


Liberia 


SG 


Singapore 







NSOOCID: <WO 0009006A1JA> 



f 



WO 00/09008 



PCT/US99/18172 



SYSTEM AND METHODS FOR CONTROLLING DEVICES BY BRAIN SIGNALS 



10 



This invention relates generally to systems, methods and devices for 
interfacing between nerve cells (neurons) and external devices and, more 
particularly, to systems, methods and devices for extracting signals directly 
from the human brain and nervous system for use in the control of external 
devices. 



BACKGROUND OF THE INVENTION 
The human brain is an exceedingly complex processing system, which 
integrates continual streams of incoming sensory input data with stored 
memories, uses the input data and memories in complex decision processes 

15 at both conscious and unconscious levels, and on the basis of these 

processes generates observable behaviors by activation of its motor or 
movement control pathways and the muscles which these innervate. 

In certain cases of traumatic injury or neurological disease, however, 
the brain is partially isolated from the periphery. Input data from certain 

20 senses are thus lost, at least for a portion of the body, as are many voluntary 

movements. Spinal cord injury is a well known example. With spinal cord 
injury, the pathways that link higher brain regions with the spinal cord and 
that are used for control of voluntary movements may be functionally 
transected at the site of injury. As a result, the patient is paralyzed, and <s)he 

25 can no longer voluntarily activate muscles that are innervated by regions of 

the spinal cord below the level of the injury. Despite the injury to their long 
fibers, however, many of the cells in these higher brain regions that control 
voluntary movement will survive and can still be activated voluntarily to 
generate electric signals for controlling voluntary movement. By recording 

30 directly from these cells with implantable devices (e.g., electrode arrays), 

signals generated by the cells may be "exteriorized" and used for the control 
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of external prostheses, such as an assist robot or an artificial arm, or 
functional electrical stimulation paralyzed muscles. 

Another example of such loss occurs in cases of amyotrophic lateral 
sclerosis (Lou Gebrig*s Disease), in which the motor neurons which control 
muscles, as well as some of the brain cells that control these motor neurons, 
degenerate. In advanced stages of this disease, the patient may have 
completely intact senses and thought processes, but is "locked in", so that 
neither movements nor behavioral expressions of any kind can be made. 
Providing these patients with some way of communicating with the external 
world would greatly enhance their quality of life. 

In sum, there is a need to develop a system for monitoring and 
processing the electrical signals from neurons within the central nervous 
system, so that the brain's electrical activity may be "exteriorized" and used 
for the voluntary control of external prostheses or assist devices. In this way, 
damaged pathways are circumvented and some control of the environment 
can be restored. Because the electrical fields of small groups of neurons 
drop off rapidly with distance from the cells, this system should include 
surgically implanted "tiny" electrodes or sensors, which can be placed in 
close proximity to the cells that generate command signals for voluntary 
movement. 

Earlier attempts to utilize signals recorded directly from neurons for the 
express purpose of controlling external prostheses have, however, 
encountered a number of technical difficulties. A major problem is how to 
obtain stable electrical signals of sufficient amplitude for real-time control of 
an external device. Two previous approaches have been used, but neither is 
successful in this regard. 

In the first approach, microelectrodes with small tips {< 300 em sq. 
surface area) have been used, which are positioned to within 10-100 em of a 
single neuron, thus isolating its action potentials or "spikes" from that of 
other, more distant cells. In some cases, two or three adjacent neurons are 
recorded from simultaneously. In such cases, electronic devices are used to 
discriminate between the spikes of the individual cells and to sort their "spike 
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trains" into distinctly recognizable signals. One problem with this approach, 
however, is that the effective "isolation" of the spikes of only one to a few 
neurons requires that the recording electrode be positioned in close proximity 
to the neurons. Thus, to obtain such records from a sufficient number of 
5 movement-related brain cells, scores of electrodes should be implanted in the 

hope that a few of them will be in just the right position to record signals from 
one or only a few movement controlling cells. Given the required proximity of 
the electrode and the cells, there is a high probability, however, that small 
movements of the former with respect to the latter will result in signal loss 

10 either because the electrode moves slightly away from the cells of interest, or 

closer to them, resulting in cellular injury. With blood pressure induced 
pulsations of the brain within the skull, such relative movement is not only 
possible but very likely. 

In recent years, small, multichannel, micromachined (integrated circuit) 

15 electrodes have been developed for use in neural recording. Given sufficient 

recording channel density, these electrodes promised a partial solution to the 
electrode/tissue movement problem described above. If the signal was lost 
from one channel by electrode movement, there was hope that it might be 
"picked" up by an adjacent channel, which moved closer to the active neuron 

20 at the same time that the previous one moved away. However, problems 

have been encountered with these electrodes as well. For reasons that are 
not entirely clear, neural signals are lost from these electrodes over time, due 
apparently to the formation of polarization potentials at dissimilar metal 
junctions along the recording channel, or the ensheathment and thus 

25 biological insulation of the electrode by glial cells. 

A second approach is to use electrodes with larger exposed recording 
surfaces (in the range of 0.5 to 1.5 mm sq. surface area). These low 
impedance electrodes have lower noise characteristics than those with 
smaller tips, and can reliably record the activity of hundreds to thousands of 

30 neurons at greater distances than can the latter. Indeed, low level 

electroencephalographic (EEG) or field potentials can even be recorded from 
the surface of the scalp. This approach thus can avoid the difficulty of 
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different signal output levels caused by small movements between the 
electrodes and the selected cells encountered in the first approach. The use 
of the signals recorded in the second approach presents, however, a major 
problem for prosthesis control. In such recordings, the desired control 
5 signals may be of very low amplitude and may be "buried" within, or 

confounded by, EEG potentials from neurons that are not involved in 
voluntary motor processes. Thus, averaging must be used over many 
movement attempts to extract a usable signal. For this reason, this approach 
is less than desirable and perhaps not useful for real-time neural control of an 

10 external device. 

Another problem, which occurs regardless of the electrode type used, is 
that neural signals may change over time for a variety of reasons: e.g., (a) 
naturally occurring cell death, which occurs randomly throughout the brain in 
adults; or (b) learning processes, which may, over time, alter the quantitative 

15 relationship between a neuron's activity and the external parts of the body to 

which it contributes voluntary control. 

SUMMARY OF THE INVENTION 
The present invention overcomes some of the difficulties described 
above with improved as well as new systems, methods and devices for 
obtaining signals directly from the brain or central nervous system, and for 
processing and utilizing these signals to control external devices. The 
systems described here are adaptable to a variety of signals from the brain or 
central nervous system as diverse as a) neurally generated electrical signals, 
recorded with microelectrode technologies from within the brain or with 
surface electrodes from extracranial sites; and/or (b) measures of localized 
blood flow that are correlated with neural activity, if techniques for 
miniaturization of current devices for making such measurements, in real 
time, are developed in the future. The external devices may include any 
device that can be controlled by processed electrical signals. These devices 
include, but are not limited to, artificial or prosthetic limbs; computer 
controlled, functional electrical stimulation of muscles of paralyzed individuals 
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for the restoration of movement; robots or robotics components; computers or 
computer displays; or the teleoperation of robots and machines in hostile 
environments. 

A preferred embodiment of the invention represents a unique blend of 
5 technologies from the fields of neuro- or electro-physiology, biomaterials 

science, neural signal processing, functional brain imaging <to guide 
implantation of sensors), and robotics or prosthetics. Included in the 
embodiment is a unique recording arrangement with bundles of six to ten 
small (20-50 urn in diameter), insulated, and flexible, noble metal wires that 
10 are arranged in a parallel or twisted array. The wire bundles are constructed 

so that each recording wire can collect multicellular signals from a small 
cluster of neurons, with tips that are incremental in length, so that many 
recording sites can be sampled along a single line of bundle insertion into the 
brain. 

15 According to another preferred embodiment, software routines, together 

with corresponding hardware, are used to perform specific signal correlation, 
adaptation, and distribution as part of a general recalibration procedure. A 
unique signal processing method is provided to convert recorded neural 
signals into a resultant signal that is useful for control of an external device. 

20 And, for the first time, the disclosed system incorporates neural net software 

routines to map actual neural signals onto desired movement functions. 

In accordance with a preferred embodiment of the invention, a robot 
arm is controlled by the neural signals recorded directly from the voluntary 
movement (motor) control areas of the cerebral cortex of a subject, such as 

25 an alert monkey. It can be appreciated, however, that the concepts and 

general procedures of using neural signals to control movements of a robot 
arm, as described in the various preferred embodiments of this invention, are 
valid for the control of any external device that can be manipulated directly or 
indirectly by electrical signals and are not limited to use with monkeys or 

30 other trained animals. The followings are the devices and stages of signal 

processing system implementation in a particular preferred embodiment of 
the invention. 
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( 1) Sensors (e.g., electrodes) and Sensor Arrays 

Sensors and sensor arrays used in a preferred embodiment of the 
present invention should have appropriate geometry and size for the 
configuration of the neural implantation site, and the methods of construction 
of the wire bundles used in this embodiment allow such flexibility. In the 
present invention, these sensors are comprised of miniature, multichannel 
microelectrodes, either (a) fabricated with micromachining/photolithography 
methods, or (b) from noble metal (e.g., platinum or Pt) microwires, oriented 
along a single axis in a parallel or twisted array. For example, for extraction 
of signals from the "arm" control regions of the cerebral motor cortex of a 
human for the purpose of rudimentary control of a robot arm, an estimated 
minimum of 16-24 recording channels would be needed, with an electrode 
density of about 24/cm sq. Further/the implanted electrodes and arrays 
should be tailored to the required geometry of the implantation site, so that 
the electrode recording tips contact the correct cellular regions. The precise 
locations and shapes of the correct cellular regions can be determined prior 
to implantation by magnetic resonance imaging (MRI) and other brain 
imaging procedures. Multisite recording from cells with these regions is best 
accomplished by a wire bundle electrode array, with the recording tips of the 
bundled wires located at 0.5-0.6 mm intervals along the shaft of the array. 
When inserted normal to the cortical surface and aligned with cortical lamina 
V in the anterior bank of the central sulcus, this electrode array can sample 
activities from several clusters of the cortical output neurons that would 
normally control arm/hand movements. Because the wire bundle electrodes 
are made of a single noble metal, they do not suffer the loss of signals due to 
polarization potentials at dissimilar metal interfaces, as occurs in the 
micromachined electrodes constructed to date. Moreover, because the 
exposed (non-insulated) surface area of the wire electrode recording tip (400- 
1800 urn sq.) is larger than that typically used in an electrode designed for 
isolation of single neurons (150-300 (im sq.), signals can be recorded with 
the tips at a greater distance from the cells of interest, thus reducing the 
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probability of cellular injury by relative movements of the electrode with 
respect to brain tissue. 

(2) Electronic Microchips 

According to one aspect of the preferred embodiment, electronic 
microchips are needed for various purposes, such as amplification, filtering, 
multiplexing, and radio transmission of signals to external receivers. 
In the examples given below, preamplification of multiple channels of neural 
data, multiplexing of neural data from multiple channels into a single data 
stream, and radio transmission out of the body of a subject would be 
accomplished by chips coupled to the electrode array, that are on the order of 
2 x 2 cm sq. or less in size. 

(3) External Receivers and Demultiplexers 

According to another aspect of the preferred embodiment, external 
receivers and demultiplexers are needed for allowing the exteriorized data 
stream to be detected and reseparated into separate neural data channels, 
which can then be individually selected and mathematically transformed for 
control of specific devices (e.g. an arm prosthesis) or device components 
(e.g., separate parts of the arm prosthesis). 

(4) Signal Usage, Correlation, and Adaptation Methods 

Once the neural signals are exteriorized, the most appropriate use for 
each should be determined. For example, if signals are derived from the 
"arm 11 area of the motor cortex, some of the signals are related to shoulder 
movements, others to movements about the elbow, some others to 
movements about the wrist, and the rest to finger movements. (In the 
paralyzed individual, of course, the movements are "attempted" rather than 
actual voluntary movements.) If the signals are not derived from the motor 
cortex but are still under voluntary control by the subject, and are to be used 
for control of arm prosthesis movements, a determination should also be 
made as to which are the most useful signals for fine hand control, which for 
upper arm control, and so forth. 

Recorded neural signals may also change over time, due to electrode 
"drift" within the brain tissue (so that the electrode moves away from some 

7 



.0009008A1JA> 



WO 00/09008 



PCT/US99/18172 



cells and closer to others), natural death of cells (50,000-100,000 cells/day in 
the adult brain), or changes in the parameters of cell discharge in relation to 
movement which result from motor learning. Consequently, a procedure 
according to the invention adapts to, or partially compensates for, changes in 
5 recorded neural signals and their parameters over time, so that the same 

implanted electrode array is useful for control purposes for as long as 
possible. 

A general neural prosthesis calibration procedure, including newly 
conceptualized signal correlation, adaptation, and distribution (software) 

10 routines, accomplishes all of these goals. In this procedure, the subject 

observes a set of simulated arm movements (or other activities) on a 
computer screen, and attempts to "track" or emulate these same movements 
with attempted movements of his/her own (paralyzed) arm. During the 
emulation process, the activity on each neural channel is cross-correlated 

15 with selected parameters of the simulated motion (joint rotation, position of 

the hand in space, speed of movement, etc.), to determine with which it is 
most highly correlated. The activity of this most correlated channel is then 
"routed" or "distributed" to the circuit which controls the device component or 
movement parameter with which it is most highly correlated. Since this 

20 calibration procedure can be performed readily by the subject or patient on a 

periodic basis, the optimal neural channels for control of the external device 
can be reselected and redistributed each time, thus providing a continuing 
adaptation to changing neural signals, regardless of the cause of these 
changes. 

25 (5) Shaping of Signals for Optimal Device Control 

Once a subset of signals has been selected for control of an arm 
prosthesis (or other device), according to one aspect of the preferred 
embodiment, the signals should still be mathematically combined for optimal 
control of that device. For example, if neural signals "x and y" out of some 

30 larger set of signals correlate well with attempted hand movements about the 

wrist, and they are to be used for that purpose in controlling an arm 
prosthesis, they should still be mathematically processed to optimize such 
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control. By "optimal" it is meant that the processed signals should drive 
movements about the artificial wrist that match as closely as possible those 
model movements that are observed by the subject during the calibration 
procedure. To accomplish this, software networks are used for mapping 
actual neural signals onto desired movement functions; in the example 
referred to here, these would be the simulated arm/hand movements 
observed by the subject on the computer screen. 

Accordingly, it is an object of the present invention to provide an 
improved system for recording neuronal signals directly from the brain or 
central nervous system for external device control 

It is another object of the present invention to develop an improved 
system which can be used with signals derived from small sets of selected 
single neurons, whose signals are separated from those of other nearby cells 
with electronic "spike" discrimination methods and are then recombined into a 
composite control signal 

It is a further object of the present invention to provide an improved 
system which can also be used with signals derived from small sets of 
neurons (2-10), whose signals are simply summed and electrically processed 
to obtain a useful signal for external device control. 

It is yet another object of the present invention to provide multi- 
electrode sensors for recording from small clusters of neurons and providing 
signals that are midway between those obtained with the single neuron and 
EEG recording approaches described above. 

It is yet a further object of the present invention to provide multichannel 
connectors and recording systems for use with the multi-electrode sensors. 

It is also an object of the present invention to provide data reduction 
methods for converting the electrical activity signals recorded from clusters of 
neurons into useful intermediate control signals. 

It is also another object of the present invention to provide an 
application of Neural Net algorithms for relating the intermediate control 
signals to movement parameters. 
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It is also a further object of the present invention to provide a novel 
calibration and adaptation method for periodically rematching recorded 
signals to the desired movements of an external device, so that 
compensation occurs for fluctuating signal amplitudes and/or motor learning. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The accompanying drawings, which are incorporated in and form a part 
of the specification, illustrate preferred embodiments of the present invention 
and, together with description, serve to explain the principles of the invention. 
In the drawings: 

FIG. 1 is an exemplary diagram of a system according to the present 
invention for use by a subject with a spinal cord injury in voluntary controlling 
an external device. 

FIGS. 2 (A) to (C) are schematics of sensors (electrodes) and sensor 
arrays according to the present invention for recording brain activity. 

FIGS. 3 (A) and (B) are photomicrographs of recording arrays of the 
present invention used to monitor the activity of sets of cortical neurons. 

FIGS. 4 (A) to (C) are diagrams of other embodiments of electrode 
arrays and methods for bringing power to and taking neural signals from an 
implantable microchip. 

FIG. 5 is a diagram of an implanted device for externalizing neural 
signals for control of prosthetic devices. 

FIG. 6 is a diagram of an implanted device according to a second 
embodiment for exteriorizing neural signals. 

FIGS. 7 (A) and (B) are waveforms schematics of signals in the second 
stage of processing of recorded multiunit neural signals. 

FIGS. 8 is a block diagram of a second stage of processing. 

FIGS. 9 (A) to (E) are waveforms schematics of signals arising in the 
processing of multiunit recordings from the arm area of the sensorimotor 
cortex of a monkey. 
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FIG. 10 is a schematic of third and later stages of neural signal 
processing system, and an example of operation of the neural prosthesis for 
control of a prosthetic or a robot arm. 

FIG. 11 is a diagram of operations performed by a movement calibration 
and signaldistribution system depicted schematically in FIG. 10. 

FIG. 12 is a schematic of an artificial neural net used to process neural 
input data before passing it to a peripheral device controller. 

FIGS. 13 (A) and (B) are graphs showing examples of processed neural 
discharge and the real-time prediction of wrist position from this neural 
discharge by the artificial neural net of FIG. 12. 

FIG. 14 is a flow chart showing steps used in calibrating the neural 
control system of FIG. 1 , using as an example the control of an artificial limb 
or functional electrical stimulation of paralyzed muscles to produce a 
particular movement. 

DETAILED DESCRIPTION OF THE DRAWINGS 
Reference will now be made in detail to preferred embodiments of the 
invention, non-limiting examples of which are illustrated in the accompanying 
drawings. 

A. Overview 

With reference to FIG. 1 , a system according to a preferred embodiment 
of the invention uses signals from cells in higher brain regions 1 can be 
activated voluntarily by a patient/subject, to in turn control external devices 
such as an artificial arm 200 and electrical stimulation of paralyzed muscles 
of an injured arm 202. FIG. 1 shows sections through various levels of the 
cerebrum 190, brain stem 192, and spinal cord 194. Also shown are 
pathways 196 that link the motor (movement control) areas of the cerebral 
cortex 198 with the spinal cord motor neurons 197 that control muscles of the 
arm 202. By recording directly from the cells in higher brain regions 1 with 
implantable devices 5 (e.g., electrode arrays), signals collected at the cells 
can, after processing by computer and electronic interfaces 9, be 
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'Exteriorized 11 and used for the control of external prostheses, such as the 
artificial arm 200 (external pathway I) or the electrical stimulation of paralyzed 
muscles 202 (pathway II). 

Alert monkeys that are trained to make specific arm movements in order 
5 to receive small food or juice rewards are used as subjects. Under surgical 

anesthesia and sterile conditions, arrays 5 of microelectrode sensors are 
implanted into the motor (voluntary movement control) areas of the animal f s 
cerebral cortex 198, opposite the trained arm, preferably according to one of 
the methods as demonstrated in FIGS. 5-6. When the animal is fully 

10 recovered, signals from sets of neurons whose electrical activity is correlated 

in time with performance of these arm movements are identified, and the 
signals are processed mathematically for control of the robot arm as shown in 
FIGS. 7-1 1 . This processing is such that the neural signals that are selected 
for control of a particular part of the arm (e.g., movements about the elbow) 

15 will, when applied to the controller for that part, produce robot arm 

movements that are very similar to those made by the animal. These "off- 
line" simulations allow of adjusting neural net parameters for optimal control 
of the robot arm as shown in FIG. 12. Again using positive reward 
procedures, the animals are then trained in 1 hour long daily sessions to 

20 adapt to the movements of the nearby robot arm, which their brain signals 

now begin to move roughly in parallel with their own arm movements. 
Delivery of the food rewards is then gradually made contingent not upon the 
accuracy of the animal's own arm movements, but upon those of the robot 
arm. Eventually, the animal's own movements can be temporarily restrained 

25 (by securing the sleeve of its training jacket), and he/she can obtain food 

solely through directly controlled movements of the robot arm with the neural 
signals recorded directly from his/her brain. The systems, methods, and 
devices according to various embodiments of the current invention can also 
be used by medical use with humans. 
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B. Sensors/Electrodes for Recording Neural Signals 

Examples of the implantable devices 5 are shown in FIGS. 2 (A) to (C). 
FIG. 2(A) shows a preferred embodiment of sensors or a sensor array 5' 
according to the present invention for recording from cortical cells of a brain 
5 for extended periods of time. Six to ten small (25 to 50 urn diameter), 

insulated, noble metal, preferably Pt or Gold, wires 12 are bundled in a 
parallel or twisted array 10, with staggered length so that the exposed 
recording tips 14 end at different cortical depths (from the surface of the 
brain) when inserted into the brain. 

10 In one embodiment of the invention, the wires 12 are constructed of 25 

pirn diameter Pt, insulated by a thin layer of Formvar, Isonel, or some other 
suitable material 15 obtained from the California Fine Wire Co. of Grover 
Beach, California. A completed wire bundle 10 made with 25 pirn wire is 
approximately 75 ^m in diameter. In another embodiment of the invention, 

15 the wires 12 are constructed of 50 ^m diameter wire. A completed bundle 10 

made with 50 |im diameter wire is about 150 ^m in diameter, and is 15-35 mm 
long from electrode tip 14 to a connector assembly 50. Electrodes made with 
the 50 jim diameter wire isolate the electrical activities of small clusters of 3 
or more neurons as well as those obtained with the 25 urn diameter wire. 

20 The wires 12 are held in position with respect to one another by a light- 

cured optical cement bead 16, placed 6-8 mm from a most distant wire tip 
142. The recording tips 14 of the individual wires 12 are located from 2 to 6-8 
mm distal to the bead 16. The difference in length from one tip to its neighbor 
tip is substantially equal in one embodiment of the invention, about 0.5 to 0.6 

25 mm, but these distances can be varied, depending upon the implantation site 

and its geometry. The bead 16, in one embodiment of the invention, is a 
small (0.5-0.6 mm diameter) drop of UV-curable acrylate, cured with 30 sec of 
UV exposure from a small, focused, UV gun. The cement bead 16 is also 
used for handling and manipulating the array 5' during its insertion into the 

30 brain, because of the bead's degree of rigidity. On the proximal side of the 

bead 16, the wires 18 separate to bundle loosely to the implanted connector 
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assembly SO (see below). By inserting several staggered arrays 5' along the 
central sulcus of the cerebral cortex 198, where major motor areas exist, it is 
possible to place them all within the cortical layer that contains the 
corticospinal neurons that participate normally in the control of voluntary 
movement. MRI is used prior to surgery to locate this region precisely in 
relation to anatomical landmarks on the skull, thus guiding electrode 
implantation. Placement of eight to twenty of sensor arrays 5' in wire bundle 
form ensures recording from sufficient numbers of cells to capture a useful 
number of control signals. 

An alternative type of sensor array 5" is shown in FIGS. 2(B) and 2(C), 
where the latter is an enlarged view of recording tips 14' shown in FIG. 2(B). 
In principle, the sensor array 5" is similar to the bundled wire array 5' shown 
in FIG. 2(A), except that in this case the entire array 5 n is constructed by 
micromachining or photolithographic techniques. In the embodiment shown, 
each recording channel or "wire" 12' is a very thin (4-10 ^m wide) strip of 
metal, left on a silicon substrate after etching away the surrounding metal. A 
long, sharp, insulating silicone substrate 11 contains six to eight separate 
recording channels 146, which end in uninsulated recording pads 144 at 
various sites along a sensor shank 13 which, when covered with a bead 16* 
of epoxy or similar material, can be used for gripping and manipulation (with 
fine tools) of the sensor array 5". Again, an enlarged bead 16' or disc region 
occurs along the shank 13. On the opposite side of this bead 16\ separate 
(insulated) wire channels 18' course to a 1 .0 mm diameter pad 50'. Though 
more desirable because of their small size and precise geometry, long-term 
recording was less successful with the micromachined sensor array 5 M than 
with the sensor array 5' in wire bundles 10 described above and shown in 
FIG. 2(A). Reasons for this include micromachined sensor's greater fragility, 
difficulty in making connections with their channels, polarization potentials 
between their dissimilar metals which block electrode conductance, the 
encasement of the electrodes by glial cells, and insulating them from neural 
signals, when they are implanted. 
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Photomicrographs of the arrays 5' and 5" are shown in FIGS. 3 (A) and 
(B) t respectively. FIG. 3(A) discloses a bundle 10 of pure platinum wires 12, 
insulated except at the tips 14. FIG. 3(B) shows a photo-lithographically 
prepared (micromachined) array 5". Thin, insulated conductors 12' travel 
from non-insulated recording pads 14' (just visible along the shank 13 of the 
electrode 5"), up a longer cable, and to a dispersed pad where electrical 
connections can be made. 

FIGS. 4 (A) to (C) show yet other types of electrode arrays that may be 
used in the present invention. In FIG. 4(A) t "n" parallel electrodes 12", each 
2.5-3.0 mm in length, protrude from a silicone wafer 20 as an insulating 
substrate, each in electrical continuity with a gold contact pad 30 on the 
opposite surface of the wafer 20. The electrodes 12" are insulated except at 
the tips 14". Each of the electrodes 12" can be made from a short length of 
90% Pt and 10% iridium wire or other noble alloy, which has sufficient 
stiffness over its short length to allow penetration of the pial covering of the 
brain and insertion into the cerebral cortex. An electrode array 25 of this type 
with each electrode 12" being a platinum plated silicon probe, but not in 
bundle form, is commercially available. As shown in FIGS. 4(B) and 4(C), an 
electrode array 25, as shown in FIG. 4(A), can be bonded to a second 
microchip 40 that contains "n" pads that mate precisely with those on the 
electrode array 25. The microchip 40 and the electrode array 25 are bonded 
in close contact and are then hermetically sealed. The microchip 40 may 
contain integrated circuits 42 that provide for amplification of the signals in 
each of the n electrodes or channels, and for multiplexing these signals into a 
serial data stream for transmission out of the brain 1 . The microchip 40 and 
electrode array 25 set upon the meninges 3 (tissue coverings of the brain 1), 
but beneath the skull (not shown), and the electrodes 12" penetrate the 
meninges 3 into the substance of the brain 1 so that neural signals can be 
directly collected therefrom. The electrodes 12" which protrude from the 
array 25 may be individual wire leads, or short bundles of the type described 
above. 
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FIGS. 5 and 6 illustrate two possible ways that the signals may be led 
from the brain 1 to external processing units. The first is a communication 
channel established by a hardware connection as shown in FIG. 5, and the 
other is a communication channel established by a wireless mechanism as 

5 shown in FIG. 6. 

Referring first to FIG. 5, in connection with FIG. 4(B), the implanted 
electrode array 35 has the first stage electronics circuitry 42 for signal 
processing (multichannel amplification and multiplexing) attached. Signals 
from the circuitry 42 are sent via a small, flexible cable 51 to a second stage 

10 device 7 implanted within the skull 2. An implant/skull junction 8 is 

hermetically sealed. This device 7 relays power from an implantable battery 
pack 56 connected to it by leads 52 beneath the skin and scalp 4. The 
implanted electronics 51, 7 and 42 are preferably programmable and contain 
circuitry for selecting (gating on) the neural channels to be used and for 

15 setting other recording parameters. They also preferably contain a 

transmitter for relaying the neural signals by RF signal to a receiver, which in 
turn relays it to demultiplexing and the later stages of neural signal 
processing described in more detail in the subsequent text and figures. 

Referring now to FIG. 6, in connection with FIG. 4(C), there is shown an 

20 alternative implant method, according to a preferred embodiment of the 

invention, which uses a wireless mechanism, thereby allowing the cranium 
(skull) 2 to be reclosed, and thus reducing chances of infection. The 
electrode implant array 45 contains the microchip 40 which has multichannel 
amplifiers, multiplexing circuitry, and an RF transmitter. Attached to the 

25 microchip 40 are two coils 53 and 54 (as shown in FIG. 4(C)). One coil 53 

allows power to be transmitted to the implant array 45 via a coil in an external 
unit 47 by induction, and the second coil 54 allows transmission of the 
multiplexed, multichannel neural signal out as a serial data stream. The 
external unit 47 contains a power coil, and a chip for conversion of DC 

30 voltages into the AC voltages that are necessary for inductive coupling to the 

internal coil 53. All devices are implanted beneath the skin. Battery packs 56 
and the external unit 47 can be changed by simple surgical procedures if 
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necessary. With the technique shown in FIG. 6, the skull 2 is dosed 
completely. While coils are used to describe the way that signals are led 
from the brain to external processing units by a nonresonant wireless 
mechanism, other kinds of devices, such as LC circuits or low pass filters, 
5 can be utilized to establish a resonant wireless mechanism to relay the 

signals as well. 

Both techniques shown in FIGS. 5 and 6 allow for complete implantation 
of the first stages of the system. In the first method (FIGS. 5 and 4(B)), a 
junction 8 is hermetically sealed between the cranial implant site 51 and the 

10 second stage device 7. In the second method (FIGS. 6 and 4(C)), the skull 2 

is resealed completely. In both cases, surgical removal and updating or 
replacement of implant components is possible if necessary. 

When these electrode arrays and devices are to be implanted in 
humans, functional MRI (fMRI) technology may first be used to identify those 

15 brain regions that the patient still has under voluntary control, thus allowing 

precise, functional placement of the implants and the recording electrodes. 

C. Second Stage Processing of Neural Signals 

The types of neural signals recorded with the electrode technologies 

20 described above are shown schematically in FIGS. 7(A) and (B). Typically, 

the recorded neural signals include action potentials or "spikes" (brief, 
voltage transients) which signal the discharge of small groups of cells located 
near the electrode recording tips 14. Because these cells are of different 
sizes and distances from the electrodes 12, 12', or 12", their action potentials 

25 will vary in shape and amplitude, and may be separated electronically or with 

computer software on the basis of these differences. FIG. 7(A) shows, for 
example, a hypothetical voltage recorded over time at one tip 14 of the 
multiwire bundle 10. The record shown in FIG. 9 can be seen to be 
composed of spikes of different amplitudes. At this stage, one of two major 

30 types of second stage processing of the recorded signals can be used. 

The first type of processing is spike discrimination method known to 
persons skilled in the art. Still referring to FIG. 7(A), an example is shown in 
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an upper trace 60 where two distinct trains 61 and 62 of action potentials, 
from two different neurons or single "units" as they are often called by 
persons skilled in the field, are interleaved in the single electrode recording. 
The single unit spike trains 61 and 62 can be separated electronically on the 
basis of differences in their spike waveforms or amplitudes, and the 
discharge frequency of each neuron (frequency = 1 /interval between spikes 
or action potentials). This separation yields two or more useful control 
signals, as shown in the lower traces 63 and 65 of FIG. 7(A), from a single 
electrode. Such spike train separation is, however, expensive in terms of 
needed hardware and software, thus raising the cost, bulk and complexity of 
any surgically implanted device. Moreover, single neurons may be damaged 
by small movements of the electrodes 12, 12', or 12 M , and their signals may 
be lost. Since the brain 1 can move slightly within the skull 2, particularly in 
relation to respiratory or cardiovascular induced changes in intracranial 
pressure, such relative movements can easily occur. Thus, while the spike 
discrimination method is one in current use today by many neurophysiological 
investigators, there have been as yet no published solutions to the problems 
of mechanical recording stability. 

To circumvent the problems of mechanical recording stability, a new 
type of second-stage processing of recorded neural signals has been 
developed as demonstrated in FIG. 7(B). In this new method, the exposed 
surfaces of the recording electrode tips 14 are made slightly larger (400-1800 
urn sq.) than is optimal for the isolation of single cell discharge, so that the 
spikes of several neurons are picked up simultaneously at each electrode tip 
14. Furthermore, to reduce the possibility that the larger spikes of very 
nearby cells will dominate the record, only to be later lost through damage to 
the cells by the electrode 12, 12\ or 12", the voltage signal is converted 
logarithmically. This conversion de-emphasizes the voltage contribution to 
the multi-unit record from nearby cells, and emphasizes that from the more 
numerous, more distant cells. The signal is then electrically rectified, yielding 
a signal 64 like that shown in the top trace of panel B. This trace 64 is 
actually the log rectified representation of the trace 60 in panel A. Thus, the 
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resulting signal S(t) 64 is given by: S(t) = log|V(t)|, where |V(t)| is the absolute 
value of the recorded multiunit voltage signal 60. Note that S(t) 64 is a 
positive function of both the log amplitude and the frequency of occurrence of 
the neural spikes. It is thus a measure of the density of neural activity in the 
vicinity of the electrode tip 14, a signal that tends to be more stable over time 
than the single unit method of recording described in the first method of 
processing neural signals. 

Next, to convert this signal 64 to one that is more useful for control of 
some external device, S(t) 64 is integrated over a short time period to yield a 
signal 66 whose peak values correspond to the integral of activity over the 
preceding time interval (see FIG. 7(B)). A 33 msec integration interval is 
chosen because it is synchronized with the video recording frame duration, 
and it is found to be useful for experimental purposes. In practice, it should 
be a period of no longer than (\l2f) t where / is the highest frequency 
component of the device movement speed that one wishes to generate. For 
human natural movement, f is on the order of 5 Hz. Thus, the integration 
period should not exceed (1/10) = 0.1 sec = 100 msec. At the end of each 
integration interval (33-100 msec), the peak value 67 of the integral 66 is 
then "read" by an electronic sample and hold (S-H) circuit, which is known to 
persons skilled in the art, the integrator is reset to zero, and another period of 
integration begins. The S-H circuit holds its value until the end of the next 
interval, and then jumps instantaneously to the new reading. Smoothing of 
the S-H output provides an analog signal 68 that is "smoother" than the 
original neural signal 60 yet still proportional to S(t) 64. Moreover, the analog 
signal 68 is much more useful for external device control because unwanted 
signals outside the frequency range of interest have been eliminated. 

An example of the second stage of signal processing is shown in FIG. 
8. During this stage of the processing, the recorded neural signals 60 having 
spikes are first converted logarithmically and rectified by the voltage 
converter 82 to yield a signal 64. Then, the signal 64 is integrated by an 
integrator 84 over a time period of no longer than (1/2/) to produce an integral 
66, where the integrator 84 is connected to the voltage converter 82 as well 
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as a S-H circuit 86. The S-H circuit 86 receives the integral 66, reads the 
peak value of the integral 66, and provides an analog signal 68 after 
smoothing. 

As an illustration, an example of these operations performed upon 
actual neural recordings from the brain of an alert monkey is shown in FIGS. 
9(A) to (E). The trace in FIG. 9(A) shows the log amplitude of the multi-unit 
activity 160 recorded from a cluster of neurons in the sensorimotor cortex of 
the monkey, during flexion (161-163 ) of his elbow as shown in FIG. 9(E); 
note the increased activity during elbow flexion 162. The trace 164 in FIG. 
9(B) shows the absolute or rectified value of the signal 160. The trace 166 as 
shown in FIG. 9(C) shows the output of the short interval, resettable 
integrator, and the trace 168 as shown in FIG. 9(D) shows the output.of the 
S-H circuit that "reads'* the value of the integrator at the end of each short 
integration period. The recordings are from experiments with a trained 
monkey, carried out during development of the technologies disclosed here. 
The time/voltage calibrations are 0.1 sec and 50 microvolts. 

D. Third and Later Stages of Signal Processing 

A summary of the third and subsequent stages of signal processing is 
shown in FIG. 10 according to a preferred embodiment of the present 
invention. 

The processed signals 68 from the S-H circuit 86 are digitized by an 
analog/digital (A/D) signal converter 100 and then fed in parallel to (a) a 
multi-channel display system 102, and (2) a signal /movement cross 
correlation and distribution system 101. The display system 102 is an n-cell 
matrix, where "n n is an integer and each cell 104 in the matrix 102 represents 
a separate neural data channel. The system 102 provides a quick overview 
of the pattern of activity across the "n" selected neural recording channels for 
each of the integration periods described above, and is extremely useful for 
initially 'focusing in" on those channels that are most active during particular 
actual or attempted movement. The display system 102 includes a LED 
display, where the intensity of the LED lighting is a positive function of the 
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level of neural activity in that channel, or a small computer screen which 
displays the ongoing level of activity in each channel in the form of color- 
coded values. Typically, the range of neural activity is divided into eight 
levels for monitoring purposes, though it is actually digitized for subsequent 
processing with 12-bit accuracy. 

The signal/movement cross correlation and distribution system 101 is 
preferably a "C" language software routine which performs the following 
operations as illustrated in FIG. 1 1 according to a preferred embodiment of 
the present invention: 

(1) Determination of mean signal levels 1 12 on each channel when the 
subject is at "rest"; 

(2) Subtraction of the mean resting signal level 1 12 for each channel 
from the ongoing signal 1 10 of that channel, thus yielding a time-dependent 
signal D(t) 114 that departs from zero only when a movement is attempted; 

(3) Computation of the cross correlations between D(t) 114 and the 
computerized display 1 16 of model arm (or joint) position, velocity, or 
acceleration (see the correlation table in step 2, FIG. 1 1) as observed by the 
subject during the calibration procedure described in detail below; 

(4) On the basis of these correlations, selection of the channels 1 18 
that will be most useful for control of movements or movement-related 
parameters of the external device; 

(5) Routing/distribution of the selected channels 1 18 to the artificial 
neural nets (shown in FIG. 12) that will "shape" that signal for control of 
particular movements or of any other external device parameter. 

Refer now back to FIG. 10. After selection of the subset 1 18 of 
channels that will be used to control a particular part of the external device, in 
this example movements about one of the "joints" in a robot arm, the signals 
on the channels 1 18 are fed to a three-layer, software or Artificial Neural Net 
(ANN) 103 for further processing, before passing to a peripheral device 
controller 107. After a succession of attempted or actual movements 105, in 
which each ANN 103 learns to "map" selected neural data channels 1 18 onto 
some measured output parameter, the ANNs 103 will thereafter shape the 
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input signals appropriately for device control. In the case of the paralyzed 
individual, the output parameter would be a desired or "calibrated" movement 
of the arm, or movement about a major joint in the arm, that (s)he observes 
on a computer display and "tries" to emulate with his/her own paralyzed arm. 
5 This "calibration" procedure is described in more detail in the next section. 

The properties of a current ANN 103 used for this purpose are depicted 
in FIG. 12. The preprocessed neural signals 120, with resting back-ground 
levels removed, are fed from the particular neural channels 1 1 8 selected for 
control of wrist movements of a robot arm to each input node of a software 

10 ANN 103. The ANN 103 has a standard three layer, feedforward network that 

is trained using back propagation with momentum. The first layer contains 
three input nodes 122, the second, hidden layer contains five nodes 124, and 
the output layer has a single node 126 (scalar output). All of the nodes are 
fully interconnected in a strictly feedforward manner; i.e., each node in a layer 

15 is fully interconnected in a feedforward manner to every node in the next 

layer, and only to those in the next layer. For instance, each input node 122 
receives input data 120 from all of the selected channels 1 18, it then sends 
an output to each node 124 in the middle or hidden layer. Each node 124 in 
the hidden layer receives inputs from each of the nodes 122 in the input 

20 layer. In turn, all nodes 124 in the hidden layer send outputs to the single 

node 126 in the final or ANN output layer. To further smooth out data 
transients that are outside of the control frequencies of interest {0-5 Hz), each 
data point fed to the ANN 103 comprises a running average of the current 
value of the neural signal in channel "m" (at time T) plus the previous nine 

25 values of that signal (from time t-1 to t-9). This averaging period was 

determined by trial and error and may be changed to other values. 

An example of the success of this three-layer net in mapping two 
channels of neural data onto a simultaneously obtained measure of an alert 
monkey's wrist position, according to a preferred embodiment of the present 

30 invention, is shown in FIGS. 13(A) and 13(B). Referring first to FIG. 13(A), 

top two traces 130 and 132 show processed multiunit activity (outputs of S-H 
circuits) from two channels, identified as channel 3 and channel 14 
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respectively, of a 36-electrode array implanted in the left motor cortex of a 
trained monkey, recorded as the animal flexed and extended his right wrist. 
Actual wrist position is shown by a third trace 134. Channel 3 shows a peak 
change in activity leading wrist extension, with a smaller elevation during 
wrist flexion. Channel 14 shows a large increase in activity leading and 
during wrist flexion, and a moderate increase during extension. Referring 
now to FIG. 13(B), output 138 of the ANN 103, which in this case is the 
position of the animal's wrist predicted from activity in neural channels 3 and 
14, is shown by the heavy trace adjacent to an offset lighter movement trace 
136 which shows his actual wrist position. Note the good correspondence, 
even though only two neural channels were used to compute predicted wrist 
position 138. The movement trace 136 is not the same as that shown in the 
previous trace 134, but was computed and recorded later, during training of 
the ANN 103. After the ANN 103 was trained over 10 movement trials, the 
match between actual and predicted wrist position attained a correlation of r = 
0.96. Use of the ANN 103 output 138 to drive a robot "wrist" produced a 
robot wrist movement trace indistinguishable from the ANN 103 output 138. 

Finally, referring back to FIG. 10, the output 138 of each neural net 103 
is fed to an interface controller 107. An interface controller is a device which 
converts the output signals from one system into an appropriate set of signals 
for controlling some other system or device and known to persons skilled in 
the art. Interface controller 107 comprises a microprocessor which is 
programmed to convert the output of the neural net 103 into the voltages and 
currents that is necessary for actuating, for example, the robot controller, and 
thus moving the robot's wrist. 

E. Calibration and Use of the System 

FIG. 14 shows steps that are followed by a paralyzed individual in 
setting up, calibrating, and using a neurally control external device according 
to a preferred embodiment of the present invention. Here the control of an 
artificial limb, or of stimulation of paralyzed muscles of the limb, to produce a 
particular motion is used as a particular example. It is assumed in this 
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description that two prior conditions have been met. First, the electrodes 
have been implanted in brain regions that have been shown with functional 
imaging procedures to be under the subject's voluntary control; i.e., which are 
"activatable" by the subject when desired. If possible, they should also be 
implanted in a brain region that is normally activated when the subject 
attempts one or more of the particular movements that the prosthetic device 
will be asked to generate, though this is not an absolutely necessary 
requirement. Second, a prosthetic specialist will have already set channel 
gains and other parameters after device implantation. Each day or at 
selected intervals thereafter, the patient/subject would perform the following 
calibration routine. 

First, (s)he learns to use a particular, monitored biological signal to turn 
on the system calibration computer when desired. If eye movements are 
intact, this signal could be a patterned sequence of eye blinks, sensed by a 
small monitoring device, that would not occur normally. If control of certain 
neck or facial muscles remains, their electromyographic activity could be 
used in the same coded way. In a totally paralyzed patient, the signal could 
be a particular time code of brain activity, generated voluntarily by the patient 
but unlike that which would occur during normal operation of the external 
device. A similar set of signals can be used for turning the calibration system 
off, or later, for turning the device to be controlled on or off when the neural 
control system has been calibrated. 

Second, with the assistance of another person or again using a coded 
sequence of biological signals, the subject selects a particular movement for 
calibration from a predetermined list. The list may have the following options: 
alternate flexion and extension of the elbow; flexion-extension of the wrist; 
movements about the shoulder; grasping and releasing of an object; reaching 
to different points in space; or some combination of these. 

When the subject gives a "ready" signal, the selected movement is 
displayed on a video monitor, simulated by animation or a video record of a 
model performing an actual movement. The model movement is performed at 
a slow to moderate speed and, during its performance, the subject "tracks'* 
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the observed movement by attempting to move his/her own paralyzed limb in 
exactly the same manner and at the same speed. "N/2" repetitions would be 
performed, where N is a non-zero even integer. During these repetitions, the 
system (a) cross correlates the neural signals with the model movement(s) or 
some selected parameter (e.g., position, velocity, acceleration) of that 
movement; and (b) on the basis of the average of these correlations over the 
N/2 repetitions, determines which subset of neural channels was most highly 
correlated with the model movement and its parameters. The activity on the 
selected subset channels would then (c) be routed to the ANNs 103 which 
control the components of the external device that will produce a movement 
like the model movement. Another N/2 repetitions are then performed, and 
the appropriate ANNs 103 (d) "map" these selected neural inputs onto a 
stored record of parameter values for the model movement. 

The subject now performs this same procedure for the next movement 
in the set, and the four steps (a) through (d) are repeated for that movement 
And so on, until the entire calibration procedure has been performed. This 
procedure is performed daily or only at needed intervals (when the subject 
notices a diminution in control accuracy), with the optimal neural channels 
being re-selected at each calibration procedure and mapped again onto the 
desired movement functions. Thus, this periodic calibration procedure 
adapts to or allows compensation for changing neural signal parameters, and 
it ensures the optimal selection of those channels that are still useful at any 
time for device control. 

Please note also that the subject's own brain can also adapt to 
changing signal properties and the challenges that these changes impose on 
device control. That is, if the subject can voluntarily activate the brain region 
from which signals are monitored and can vary these signal levels, then (s)he 
can learn to modulate these activation levels so that the external device can 
still be manipulated, even if there is drift or other unknown changes in the 
activity of the recorded neural channels. 

Obviously, many modifications and variations of the present invention 
are possible to persons skilled in the art, without departing from the spirit and 
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CLAIMS 

What is claimed is: 

1 . A device for collecting multicellular signals directly from a nervous 
system and transmitting the signals to an external receiver, comprising: 

5 A) a plurality of electrodes formed in bundles of flexible wires with 

tips at staggered length aligned to be implanted in the nervous system for 
collecting multicellular signals; and 

B) a signal processing mechanism connected to the electrodes for 
multiplexing and transmitting the signals from the electrodes to the external 
10 receiver. 

2. The device according to claim 1 , wherein the number of the flexible 
wires in a bundle ranges from 3 to 10. 

3. The device according to claim 1 , wherein the diameter of the flexible 
wires is smaller than 50 em. 

15 4. The device according to claim 1, wherein the difference in length from 

one tip to its neighbor tip of the wires is variable. 

5. The device according to claim 4, wherein the difference in length from 
one tip to its neighbor tip of the wires is about 0.3 to 0.6 mm. 

6. The device according to claim 1 , wherein the flexible wires in a bundle 
20 are made from noble metal. 

7. The device according to claim 1 , wherein the signal processing 
mechanism comprises: 

A) an insulating substrate; 

B) an array of electrode contact pads mounted on the substrate for 
25 receiving the electrodes, each electrode received individually by one of the 

electrode contact pads, and with the electrode contact pads being electrically 
isolated from each other at the substrate; 

C) a microchip bonded in close contact with the substrate for 
receiving and multiplexing the signals from the electrodes; and 

30 D) an electric connection to the microchip for relaying power to the 

microchip and transmitting the signals received at the microchip to the 
external receiver. 
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8. The device according to claim 7, wherein the electric connection is a 
hardware device. 

9. The device according to claim 8, wherein the hardware device is a 
cable. 

10. The device according to claim 7, wherein the electric connection is a 
wireless device. 

1 1 . The device according to claim 1 0, wherein the wireless device is a 
nonresonant wireless setup. 

12. The device according to claim 1 1 , wherein the nonresonant wireless 
setup comprises: 

A) a first coil in connection with the microchip for relaying power to 
the microchip; and 

B) a second coil in connection with the microchip for receiving and 
transmitting the multiplexed signals to the external receiver. 

13. The device according to claim 10, wherein the wireless device is a 
resonant wireless setup. 

14. The device according to claim 13, wherein the resonant wireless setup 
comprises: 

A) a coil in connection with the microchip for relaying power to the 
microchip; and 

B) a low pass filter in connection with the microchip for receiving 
and transmitting the multiplexed signals to the external receiver. 

15. The device according to claim 7, wherein the microchip comprises 
signal processing integrated circuits. 

16. An apparatus for recording multi-neuron signals directly from a nervous 
system comprising a plurality of sensors, wherein each sensor further 
comprises a bundle of noble metal wires with tips at staggered length. 

17. An apparatus for recording multicellular signals directly from small 
clusters of neurons and broadcasting the signals to an external receiver, 
comprising: 

A) an array of electrodes for receiving the signals; 
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B) a signal processor for multiplexing and transmitting the signals 
to the external receiver, wherein the signal processor comprises: 

i) an insulating substrate; 

ii) an array of electrode contact pads mounted on the 
substrate for receiving the electrodes, each electrode received individually by 
one of the electrode contact pads with the electrode contact pads being 
electrically isolated from each other at the substrate; 

iii) a microchip bonded in close contact with the substrate 
for receiving and multiplexing the signals from the electrodes, wherein the 
microchip comprises signal processing integrated circuits; and 

iv) a electric device connected to the microchip for relaying 
power to the microchip and carrying the signals from the microchip to the 
external receiver. 

18. The apparatus according to claim 17, wherein the electrodes are formed 
in a bundle of noble metal wires with tips at staggered length. 

19. The apparatus according to claim 17, wherein the electric device is a 
hardware connection. 

20. The apparatus according to claim 19, wherein the hardware connection 
is a cable. 

21 . The apparatus according to claim 17, wherein the electric device is a 
wireless connection. 

22. The apparatus according to claim 21 , wherein the wireless connection 
is a nonresonant wireless setup. 

23. The apparatus according to claim 22, wherein the nonresonant wireless 
setup comprises: 

A) , a first coil in connection with the microchip for relaying power to 
the microchip; and 

B) a second coil in connection with the microchip for receiving and 
transmitting the multiplexed signals from the microchip to the external 
receiver. 

24. The apparatus according to claim 21, wherein the wireless connection 
is a resonant wireless setup. 
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25. The apparatus according to claim 24, wherein the resonant wireless 
setup comprises: 

A) a coil in connection with the microchip for relaying power to the 
microchip; and 

B) a low pass filter in connection with the microchip for receiving 
and transmitting the multiplexed signals from the microchip to the external 
receiver: 

26. An instrument for collecting multicellular signals directly from a nervous 
system and transmitting the signals to an external receiver, comprising: 

A) at least one sensor for receiving the signals, wherein the sensor 
comprising: 

i) a bundle of wires with staggered length in an array, 
wherein each wire has a tip for receiving the signals from the nervous system; 
and 

ii) an optical cement bead through which the wires may 
pass for holding the wires in position in a selected distance from the most 
distant wire tip; and 

B) a signal processor connected to the sensor for multiplexing and 
transmitting the signals from the sensor to the external receiver. 

27. The instrument according to claim 26, wherein the selected distance is 
about 6-8 mm. 

28. The instrument according to claim 26, wherein the wires are made from 
conductive materials. 

29. The instrument according to claim 28, wherein the conductive materials 
are noble metal. 

30. A method for using multicellular signals collected directly from a 
nervous system to control an external device comprising the steps of: 

A) collecting multicellular signals directly from a nervous system; 

B) multiplexing and broadcasting the signals to an external 
receiver; 

C) receiving the signals for demultiplexing and separating the 
signals; 
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D) rectifying the signals; and 

E) converting the signals logarithmically. 

31 . The method according to claim 30, further comprising the steps of: 

F) integrating the log rectified signals over a short time period; 

G) sample-hold reading of the peak value of the integral for 
generating a S-H output; and 

H) using the S-H output for controlling the external device. 

32. A channel display system for monitoring recorded multicellular signals, 
comprising: 

A) a device for recording multichannel neural signals directly from 
a nervous system and broadcasting the signals; 

B) an external receiver for receiving the signals; 

C) electric connection means for connecting to an external receiver 
to relay the signals; and 

D) a visual display for displaying the signals in a matrix, wherein 
each cell of the matrix represents a separate neural data channel. 

33. The channel display system according to claim 32, wherein the visual 
display comprises an LED, wherein the intensity of illumination of the LED is 
a positive function of the level of neural signals fed from the related neural 
data channel. 

34. The channel display system according to claim 32, wherein the visual 
display comprises a computer screen, wherein the color of each cell of the 
display matrix on the screen is coded according to the level of neural signals 
fed from the related neural data channel. 

35. A method for using multicellular signals collected directly from a 
nervous system to control a prosthetic device comprising the steps of: 

A) using a detective device with a plurality of electrodes to record 
multicellular signals directly from a nervous system; 

B) multiplexing and broadcasting the signals to an external 
receiver; 

C) processing the signals; 

D) converting the signals in digital form; and 
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E) displaying the signals in a multi-channel visual display where 
the display in one channel correlates to signals collected from one electrode. 

36. A method for using multicellular signals collected directly from a 
nervous system to control a device comprising the steps of: 

5 A) setting up a calibration routine for a subject's learning of a 

movement; 

B) developing a model for that movement; 

C) using a detective device with a number of electrodes to record 
multicellular signals directly from the subject's nervous system, wherein one 

10 electrode represents one signal channel; 

D) multiplexing and broadcasting the signals to an external 
receiver; 

E) processing the signals; 

F) converting the signals in digital form; 

15 G) determining of mean signal levels over a period of 10 to 20 

seconds for each channel where signals in one channel correlates to signals 
collected from one electrode when the subject is at rest; 

H) subtracting the mean resting signal level for each channel from 
the signals recorded at different time to yield a signal as a function of time, 

20 D(t), that departs from zero only when a movement is attempted; 

I) computing the cross correlation between D(t) and the 
computerized model of the attempted movement in terms of at least one 
motion parameter; and 

J) selecting the neural channels which are useful for controlling 
25 the device or a component of the device based on the cross correlation 

calculation. 

37. The method according to claim 36, further comprising the step of: 

K) distributing the selected channels to neural nets for shaping the 
signals for control of the movement of the device. 
30 38. The method according to claim 36, wherein the motion parameters are 

position, velocity, and acceleration. 
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39. An apparatus for collecting multicellular signals directly from a nervous 
system to control an external device, comprising: 

A) sensors in an array for recording the signals; 

B) a transmitter for broadcasting the signals; 

5 C) a processor for receiving the signals, converting the signals 

logarithmically, and integrating the signals to generate a S-H output in analog 
form; 

D) a converter for converting the signals in analog form into 
digitized signals; 

10 E) a computing interface for performing neural signal/model 

movement cross correlation to select the signals that are most useful for 
control of a particular movement of the device and distributing the selected 
signals; 

F) a microcomputer for supporting a feedforward neural network 
15 for receiving the selected signals and mapping the signals onto a model 

movement to generate control parameters; and 

G) an interface controller for controlling the device to make 
movement same or similar to the model movement according to the control 
parameter. 

20 40. A method for using a neural net to map neural signals onto desired 

movement or control parameters to control a device comprising the steps of: 

A) recording neural signals in multichannels directly from a 
subject's nervous system; 

B) removing resting background levels from the signals; 

25 C) calculating a running average for each data point of the signals 

at time T in channel u m D by combining the previous several values of that 
signal; 

D) feeding the signals from the particular channels selected 
according to the desired movement or control parameters in a feedforward 
30 direction to each input node of the neural net, wherein the neural net has at 

least three input nodes, and all the nodes are fully interconnected; and 
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F) outputting the signals from the neural net to a peripheral device 
controller. 

41. A method for calibrating a neutral control system comprising the steps 
of: 

A) turning on the control system; 

B) selecting a model movement for calibration; 

C) giving ready signal to the system; 

D) displaying the selected model movement on a visual display; 

E) allowing a subject to attempt to track or emulate the selected 
model movement; 

F) recording neural signals from a number of neural channels of 
the subject during the attempted movement; 

G) comparing the recorded neural signals with the selected model 
movement by correlation methods, so that the neural channels whose activity 
is most highly correlated with the attempted movement may be selected by 
the system; 

H) repeating steps E) to G) for N/2 times, wherein N is a number 
equal to or greater than one; and 

I) performing another N/2 repetitions of E) to G) to allow a neural 
net to map the recorded neural signals from the selected channels onto a set 
of parameters for the selected model movement. 

42. A movement calibration system comprising: 

A) a biological signal monitor for turning on or off the system; 

B) a database for storing optimal parameters related to a family of 
model movements a subject attempts to make; 

C) a visual display for displaying a movement selected from the 
database; and 

D) a computer for matching the selected movement and the 
subject's attempted movements by calculating their correlation. 

43. A method for using multicellular signals collected directly from a 
nervous system to control a prosthetic device comprising the steps of: 
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A) setting tip a calibration routine for a subject's learning of a 
desired movement; 

B) computerizing a "model" for that desired movement; 

C) using a detective device with a number (n) of electrodes to 

5 record multicellular signals directly from the subject's nervous system when 

the subject attempts the desired movement, wherein each of the n electrodes 
represents one signal channel; 

D) multiplexing and broadcasting the signals to an external 
receiver; 

10 E) processing the signals for generating signals in analog form; 

F) converting the signals into digital form; 

G) determining of mean signal levels for each channel where 
signals in one channel correlates to signals collected from one electrode 
when the subject is at "rest"; 

15 H) subtracting the mean resting signal level for each channel as 

computed in step (G) from the collected ongoing signal to yield a signal as a 

function of time, D(t), that departs from zero only when a movement is 

attempted by the subject; 

I) computing the cross correlation between D(t) and the 
20 computerized model of the desired movement in terms of kinetic parameters 

such as position, velocity, and/or acceleration; 

J) selecting the neural channels which are most useful for 

controlling the device or a component of the device based on the cross 

correlation calculation; and 
25 K) distributing of the selected channels to neural nets to shape the 

signals for controlling the device to produce a movement as same or similar 

to the desired movement 
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